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ABSTRACT 

This  research  examines  the  evaluation  performed  by  the  Navy  International  Programs 
Office  on  United  States  Navy  programs  proposed  as  candidates  for  international 
codevelopment  agreements.  Primary  research  consists  of  a  spreadsheet  analysis  of  22 
program  evaluations  conducted  by  the  International  Programs  Office  during  1990-1991. 
Secondary  research  recounts  the  history  of  the  U.  S.  Government's  international  armaments 
cooperative  efforts  from  World  War  II  to  the  present,  explores  current  issues,  and  examines 
Department  of  Defense  and  Department  of  the  Naw  policy  and  procedures  governing 
international  cooperative  programs. 
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I .    INTRODUCTION 

A.   BACKGROUND 

On  October  5,  1990,  the  Navy  International  Programs  Office 
(IPO)  instituted  a  five- step  evaluation  method  using  a  table 
of  14  success  criteria,  grouped  in  three  sections,  to  evaluate 
Research  and  Development  (R&D)  programs  proposed  as  candidates 
for  international  codevelopment  agreements  between  the  U.  S. 
Government  and  allied  nations.  Before  the  development  of  the 
success  criteria,  candidate  programs  were  qualitatively 
evaluated  using  a  lengthy  worksheet.  With  the  introduction  of 
the  success  criteria,  candidate  programs  were  evaluated  to 
identify  those  programs  with  a  low  likelihood  of  success  in 
the  international  arena,  and  so  direct  the  IPO's  resources  at 
those  candidate  programs  more  likely  to  succeed. 

The  evaluation  consisted  of  applying  the  14  success 
criteria  twice  to  each  candidate  program,  once  to  rate  U.  S. 
participation  in  the  program  and  once  to  rate  allied 
participation.  Six  strength  ratings  were  then  calculated,  one 
for  each  section  of  criteria  for  both  the  U.  S.  Government's 
and  the  foreign  participation.  These  paired  ratings  were 
compared  to  each  other;  based  on  their  "degree  of 
convergence,"  a  weighted  factor  ranging  from  0.5  to  1.5  was 
then  applied  to  each  paired  ratings.   The  three  resultant 


weighted  rates  were  added  to  arrive  at  a  final  composite 
strength  rating  for  each  candidate  program.  Eighteen 
candidate  programs  were  thus  evaluated  in  the  following 
months;  they  were  ranked  in  descending  order  according  to 
their  composite  strength  ratings. 

On  July  30,  1991,  IPO  revised  the  success  criteria, 
dropping  two  items,  substituting  in  for  two  others,  and 
regrouping  the  criteria  into  four  sections  vice  three. 
Generally,  the  five-step  evaluation  process  remained 
unchanged.  The  new  procedure  did  call,  however,  for  only  a 
single  "strength  matching"  weight  to  be  applied  to  the  entire 
evaluation  to  determine  the  candidate's  composite  strength, 
this  instead  of  the  area  "degree  of  convergence"  weights. 
Four  additional  candidate  programs  and  six  of  the  original  18 
candidate  programs  were  evaluated  using  the  revised  criteria 
and  procedures.  The  results  revealed  some  anomalies:  not 
only  did  the  four  additional  candidate  programs  score 
generally  lower  than  the  six  original  programs,  but  the 
priority  ranking  of  some  original  programs  was  reversed. 

B .   OBJECTIVES 

The  general  objective  of  this  research  was  to  examine  the 
impact  that  the  revised  evaluation  criteria  and  procedures  had 
on  the  priority  ranking  of  candidate  programs,  and  so  verify 
the  accuracy  of  the  revised  criteria  and  procedures  at 


indicating   a   program's   potential   for   success   as   an 
international  codevelopment  agreement. 

C.  RESEARCH  QUESTION 

The  primary  research  question  addressed  by  this  research 
is  as  follows: 

•  How  can  the  evaluation  conducted  by  the  Navy  International 
Programs  Office  on  programs  being  considered  for 
nomination  as  international  codevelopment  agreements  be 
refined  so  as  to  offer  a  more  accurate  "front  end" 
analysis  of  the  candidate  programs? 

Secondary  research  questions  addressed  by  this  research  in 
support  of  the  primary  research  question  are  as  follows: 

•  What  has  been  the  U.  S.  Government's  participation  in 
international  armaments  agreements  since  World  War  II? 

•  What  are  the  current  issues  affecting  efforts  at 
international  armaments  agreements? 

•  What  are  Department  of  Defense  and  Department  of  the  Navy 
policies  and  procedures  regarding  international 
codevelopment  agreements? 

D.  SCOPE,  LIMITATIONS  AND  ASSUMPTIONS 
1.   Scope 

This  research  examines  the  evaluations  performed  by  IPO  on 
22  candidate  programs  from  October,  1990  through  July,  1991. 
Data  for  this  research  consist  of  the  original  and  revised 
success  criteria,  the  18  evaluations  conducted  using  the 
original  criteria  and  procedures,  and  the  ten  evaluations 
conducted  using  the  revised  criteria  and  procedures. 


2.  Limitations 

The  only  significant  limitation  affecting  this  research 
was  an  inability  to  access  the  computer  program  that  generated 
the  evaluation  sheets  for  the  International  Programs  Office; 
access  to  the  computer  was  denied  for  security  reasons.  In 
lieu  of  access,  computer  printouts  representing  all  the 
evaluations  performed  by  the  IPO  during  the  period  of  the 
research  were  provided. 

3 .  Assumptions 

The  research  assumes  that  the  reader  has  a  basic 
understanding  of  spreadsheet  calculations  and  construction. 

E.   LITERATURE  REVIEW  AND  METHODOLOGY 

A  literature  review  was  conducted  to  determine  the  history 
of  the  U.  S.  Government's  participation  in  international 
armaments  agreements  since  World  War  II,  and  to  determine 
current  issues  affecting  international  codevelopment  efforts. 
A  review  of  relevant  Department  of  Defense  instructions  and 
Department  of  the  Navy  reports  was  conducted  to  determine 
official  policy  and  procedures. 

The  methodology  for  this  research  consisted  primarily  of 
a  personal  interview  with  Mr.  Frank  D.  Kenlon,  Director  for 
International  Programs  with  the  Navy  International  Programs 
Office  in  September,  1991.  Extensive  use  is  made  of  a 
spreadsheet  analysis  of  the  individual  evaluation  sheets. 
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F.   OVERVIEW 

Chapter  II,  Background,  recounts  the  U.  S.  Government's 
involvement  in  international  armaments  cooperative  agreements 
with  its  allies  from  World  War  II  to  present.  It  additionally 
explores  current  issues  in  codevelopment  and  coproduction 
agreements,  and  examines  current  and  proposed  Department  of 
Defense  (DoD)  and  Department  of  the  Navy  (DoN)  policy 
regarding  international  agreements. 

Chapter  III,  Presentation  of  Data,  displays  the  data  used 
in  conducting  this  research.  Tables  exhibiting  the  original 
and  revised  success  criteria,  and  individual  evaluation  sheets 
for  all  program  evaluations  conducted  by  the  IPO  are  included. 

Chapter  IV,  Analysis  of  Data,  reproduces  the  cell -by- cell 
spreadsheet  analysis  of  the  various  evaluation  sheets,  and 
correlates  the  IPO's  success  criteria  to  stated  Office  of  the 
Secretary  of  Defense  (OSD)  international  program  success 
indicators . 

Chapter  V  offers  the  conclusions  and  recommendations 
developed  as  a  result  of  the  research. 


II.    BACKGROUND 

A.   U.  S.  PARTICIPATION  IN  MODERN  MILITARY  ALLIANCES 

The  United  States'  involvement  in  its  present  day  military- 
alliances  is  largely  a  result  of  the  economic  and  military 
uncertainties  that  characterized  the  world  in  the  years 
following  World  War  II.  The  war  levied  a  terrible  toll  across 
the  Europe  and  Asia,  so  that 

.  .  .  the  global  balance  of  power  after  the  war  would  be 
totally  different  from  that  preceding  it.  Former  Great 
Powers- -France,  Italy- -were  already  eclipsed.  The  German 
bid  for  mastery  in  Europe  was  collapsing,  as  was  Japan's 
bid  in  the  Far  East  and  Pacific.  Britain  .  .  .  was 
fading  ....  Only  the  United  States  and  the  USSR 
counted,  so  it  seemed;  and  of  the  two,  the  American 
"superpower"  was  vastly  superior  (Kennedy,  357) . 

Much  of  the  United  States'  power  and  prestige  lay  in  its 

economic  might: 

The  country's  GNP  measured  in  constant  1939  dollars  rose 
from  $88.6  billion  (1939)  to  $135  billion 
(1945)  ....  industrial  expansion  in  the  United  States 
rose  at  a  faster  pace  .  .  .  than  at  any  period  before  or 
since  ....  Among  the  Great  Powers,  the  United  States 
was  the  only  country  which  became  richer- -in  fact,  much 
richer- -than  poorer  because  of  the  war  (Kennedy,  357-358)  . 

By  comparison,  the  former  Great  Powers  (Germany,  Japan,  Italy, 

France  and  Great  Britain)  all  suffered  greivous  economic 

reversals  as  a  result  of  the  war.  Germany  was  divided  up  into 

four  military  occupation  zones,  and  later,  into  two  countries; 

its  industrial  base,  devastated  by  bombing,  was  dismantled; 

its  national  income  and  output  dropped  over  two- thirds  between 


1938  and  1946  (Kennedy,  365).  Japan  was  similarly  occupied; 
its  national   income  fell  43  percent  from  1936   to  1946 

(Kennedy,  365)  .  Italy's  GNP  in  1945  was  only  60  percent  of 
its  1938  level  (Kennedy,  365-366).  France  had  been  plundered 
by  the  Germans  for  four  years  and  then  suffered  from  the  heavy 
fighting  during  1944;  its  1945  income  was  only  50  percent  of 
its  prewar  level  (Kennedy,  366).  Great  Britain  fared  better 
than  those  countries,  but,  even  so,  was  suffering  economically 
at  the  end  of  the  war;  it  had  depleted  its  gold  and  dollar 
reserves,  worn  out  its  domestic  industrial  capability,  and  was 
heavily  dependent  on  the  United  States  for  much  of  its  arms, 
shipping,  food  and  other  supplies  (Kennedy,  367) . 

The  United  States'  economic  might  was  projected  worldwide 
by  its  military  power.  At  the  end  of  the  war,  the  U.  S. 
military  was  comprised  of  some  12.5  million  personnel  (7.5 
million  deployed  overseas) ;  its  modern  weaponry  included  1200 
warships,  3000  bombers,  and  a  monopoly  of  atomic  weapons 

(Kennedy,  358).   Of  the  European  powers,  only  Great  Britain 

possessed  a  credible  military  force  at  the  end  of  the  war, 

with  forces  deployed  to 

.  North  Africa,  Italy,  Germany,  (and)  Southeast 
Asia.  Despite  heavy  losses,  the  Royal  Navy  possessed  over 
1,000  warships,  nearly  3,000  minor  war  vessels,  and  nearly 
5,500  landing  craft.  RAF  Bomber  Command  was  the  second- 
largest  strategic  air  force  (by  far)  in  the  world 
(Kennedy,  3  67) . 

If  World  War  II  heralded  the  emergence  of  the  United 

States  as  a  superpower,  the  same  must  also  be  said  of  the 


Soviet  Union.  The  increasing  Soviet  influence  can  be  seen  by- 
its  territorial  growth  during  and  after  the  war.  Its  European 
boundaries  expanded 

.  .  .  in  the  north  at  the  expense  of  Finland,  in  the 
center  at  the  expense  of  Poland;  and  in  the  south, 
recovering  Bessarabia,  at  the  expense  of  Rumania.  The 
Baltic  states  of  Estonia,  Latvia,  and  Lithuania  were 
reincorporated  into  Russia.  Part  of  East  Prussia  was 
taken,  and  a  slice  of  eastern  Czechoslovakia  .  .  .  was 
also  thoughtfully  added,  so  that  there  was  direct  access 
to  Hungary.  To  the  west  and  southwest  of  this  enhanced 
Russia  lay  a  new  cordon  sanitaire  of  satellite  states, 
Poland,  East  Germany,  Czechoslovakia,  Hungary,  Rumania, 
Bulgaria,  Yugoslavia  and  Albania  (Kennedy,  361) . 

In  the  Far  East,  the  Soviets  occupied  Manchuria,  North 

Korea  and  Sakhalin  and  linked  up  with  the  Chinese  Communists 

(Kennedy,  367) .   After  the  war,  Stalin  would  reduce  the  size 

of  the  Soviet  Army  by  two- thirds 

to  the  still  very  substantial  total  of  175 
divisions,  supported  by  25,000  front-line  tanks  and  19,000 
aircraft.  It  would  still  remain  .  .  .  the  largest  defense 
establishment  in  the  world- -a  fact  justified  .  .  .  by  its 
need  to  deter  future  aggressors  and  ...  to  keep  control 
of  its  newly  acquired  satellites  in  Europe  as  well  as  its 
conquests  in  the  Far  East  (Kennedy,  363) . 

The  Soviet  military  strength  belied  the  country's  economy, 

which  was  shattered  by  the  war.   Estimates  put  the  number  of 

Soviet  deaths  as  a  result  of  the  war  at  20-25  million;  in 

addition,   livestock,   farm  equipment,   the   transportation 

infrastructure,  and  housing  were  likewise  decimated  by  the 

occupying  German  Army  (Kennedy,  362).   In  short,  "the  Russia 

of   1945   was   a  military  giant   and,   at   the   same   time, 

economically  poor,  deprived  and  unbalanced"  (Kennedy,  362) . 


The  ideological  splitting  of  Europe  into  the  Capitalist 

Western  bloc  and  the  Communist  Eastern  bloc  became  apparent 

soon  after  the  end  of  the  War.   It  was  noted  in  1946  and  early 

in  1947  that 

.  .  .  various  plebiscites  and  regional  elections  in  the 
German  zones  (of  occupation)  were  showing  "the  political 
complexion  of  West  Germany  .  .  .  beginning  to  differ 
markedly  from  that  of  East  Germany";  the  steady 
elimination  of  any  non- Communist  elements  in  Poland, 
Bulgaria,  and  Rumania  was  mirrored  by  the  internal 
political  crisis  in  France  in  April,  1947,  when  the 
Communists  were  forced  to  resign  from  the  government.  A 
month  after,  the  same  happened  in  Italy.  In  Yugoslavia, 
Tito's  political  domination  (in  place  of  the  Allied 
wartime  agreements  about  shared  power)  was  interpreted  by 
the  West  as  a  further  step  in  Moscow's  planned  advance 
(Kennedy,  3  75) . 

A  two-pronged  strategy  was  developed  by  the  West  to 

contain  what  was  perceived  as  the  Soviet  threat  to  Western  and 

Southern  Europe.   The  first  element  of  this  strategy 

.  .  .  was  to  indicate  to  Moscow  those  regions  of  the 
globe  which  the  United  States  "cannot  permit  ...  to  fall 
into  hands  hostile  to  us."  Such  states  would  ...  be 
given  military  support  to  build  up  their  powers  of 
resistance;  and  a  Soviet  attack  on  them  would  be  regarded 
virtually  as  a  casus  belli  ....  The  (second  and)  most 
crucial  component  of  any  long-term  containment  policy 
would  be  a  massive  program  of  U.  S.  economic  aid,  to 
permit  the  rebuilding  of  the  shattered  industries,  farms 
and  cities  of  Europe  and  Japan;  for  that  would  not  only 
make  the  latter  far  less  likely  to  be  tempted  by  Communist 
doctrines  of  class  struggle  and  revolution,  it  would  also 
help  to  readjust  the  power  balances  in  America's  favor 
(Kennedy,  3  76) . 

In  Europe,   the  economic  aid  element  of  the  containment 

strategy  was  implemented  by  the  Marshall  Plan  in  1947;  the 

military  support  element  would  come  two  years  later  when, 


following  the  Berlin  crisis  of  1948-49,  the  North  Atlantic 
Treaty  Organization  (NATO)  was  created. 

B.   U.  S.  ARMS  COOPERATION  EFFORTS  SINCE  WORLD  WAR  II 

In  the  first  decade  following  the  signing  of  the  NATO 

treaty,  the  member  nations,  confronted  by  the  Sputnik  launch 

with  an  eclipsing  of  Western  missile  technology,  began  to 

cooperate  on  armaments  (Taft,  15).   An  early  expression  of 

this  cooperative  trend  was 

.  the  NATO  Co -product ion  Program  agreed  by  the 
Alliance  in  1957  ....  Since  then  there  have  been 
literally  hundreds  of  bilateral  and  multilateral  European 
and  transatlantic  cooperative  programs- -co- development , 
co-production,  licensed  production  projects,  MOU's,  dual 
production,  industry- to- industry  efforts,  outright  weapons 
sales,  one  allied  nation  to  another  (Taft,  15) . 

Three  phases  of  the  Government's  arms  cooperation  efforts 

since  World  War  II  can  be  identified.  The  first,  lasting  from 

the  end  of  the  war  until  the  mid- sixties, 

.  was  characterized  by  grant  aid  known  as  the 
Military  Assistance  Program  (MAP)  ....  the  emphasis  was 
on  the  no-cost  (to  the  recipient)  transfer  of  equipment 
directly  from  U.  S.  Force's  inventories- -initially  surplus 
stocks  developed  as  a  result  of  modernization  of  our  own 
forces- -or  from  additional  new  production  of  systems  being 
produced  for  U.  S.  Forces  (Cole,  F-3) . 

Funding  for  the  MAP  came  entirely  from  the  U.  S.  Congress;  the 

funds  were  administered  by  a  central  office,  the  Director  of 

Military  Assistance,  that  did  not  concern  itself  with  the 

implementation  of  the  programs  by  the  various  acquisition  and 

logistics  Commands  involved  (Cole,  F-3). 
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These  arrangements  worked  well.  Service  hardware 
Program  Managers  had  few  if  any  complaints.  The  equipment 
being  furnished  was  standard  U.  S.  The  requirements  were 
easily  folded  into  contracts  for  equipping  U.  S.  forces; 
the  funding  was  U.  S.  budget  authority;  and  there  were  no 
issues  of  R&D  recoupment,  administrative  surcharges,  asset 
use  charges,  agent  fees,  co-production,  offsets,  MOUs , 
etc.,  that  became  commonplace  for  providing  U.  S.  arms  to 
friends  and  allies  (Cole,  F-3) . 

The  second  phase  of  arms  cooperation  overlaps  the  first 

but  can  be  characterized  by 

...  a  gradual  shift  from  MAP  to  sales  on  a  government- 
to-government   basis   usually   referred   to   as   Foreign 
Military  Sales   (FMS) ,  and  to  direct  commercial  sales 
(Cole,  F-3) . 

The  Foreign  Military  Sales  program 

.  initially  focused  on  the  stronger  economies  of 
European  NATO  .  .  .  with  the  objective  of  having  allies 
obviously  able  to  pay  their  own  way  assume  larger  shares 
of  the  common  defense  burden. 

MAP  procedures  continued  for  most  other  countries 
friendly  or  allied  to  the  United  States  and  also  for  some 
European  NATO  allies  (Cole,  13-14) . 

Responsibility  for  FMS  was  transferred  from  the  Director 

of  Military  Assistance  to  the  newly  established  position  of 

Director  of  International  Logistics,  and  later,  along  with 

responsibility  for  MAP,   to  the  Director  of  the  Defense 

Security   Assistance   Agency   (DSAA)   upon   that   agency's 

establishment  in  1971  (Cole,  F-3  -  F-4)  .   Congress  began  to 

curtail  MAP  funds,  with  the  result  that  DSAA  attention  became 

increasingly  focused  on  FMS  transactions;  FMS  credits  became 

the  means  by  which  countries  progressed  from  MAP  to  cash  sales 

(Cole,  F-4) .   DSAA  (conceived  as  simply  the  fund  manager  and 

resident   authority   for   international   arms   programs) , 
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eventually   became   "firmly   astride   and   enmeshed   in   the 

acquisition  and  logistics  function"  (Cole,  F-4)  .   In  response, 

the 

.  .  .  OSD  acquisition  and  logistics  staffs,  the  Military 
Departments  and  Logistics  Commands  demanded  and  received 
a  greater  voice  in  international  arms  matters.  By  1976 
program  direction  had  become  diffused  and  controversies 
were  common  (Cole,  F-4) . 

A  DoD  task  force,  charged  by  Secretary  of  Defense  Rumsfeld 
with  reviewing  Security  Assistance,  recommended  on  January  14, 
1977  that  DSAA  should  report  directly  to  the  Defense 
Acquisition  Executive  vice  the  Assistant  Secretary  of  Defense- 
International  Security  Affairs,  but  this  recommendation  was 
never  acted  upon  by  the  incoming  Carter  Administration  (Cole, 
F-4)  . 

The  third  (and  current)  phase  of  the  Government's  arms 
cooperation  efforts  is  characterized  by  declining  U.  S.  arms 
exports  and  an  increase  in  the  number  of  codevelopment 
agreements  (Cole,  F-4) .  An  examination  of  this  phase  occurs 
in  later  in  this  chapter  as  part  of  the  discussion  of 
Rationalization,  Standardization  and  Interoperability  (RSI)  as 
it  applies  to  NATO. 

C.   ADVANTAGES  AND  DISADVANTAGES  OF  ARMS  COOPERATION 

The  most  obvious  advantage  to  two  or  more  countries 
entering  an  arms  cooperation  agreement  is  economic.   Thomas  A. 
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Callaghan  notes: 

For  centuries  commercial  economics  and  trade  have  been 
moving  towards  ever-larger  markets,  providing  economies  of 
scale  and  ever  more  affordable  prices  ....  Defense 
economics  has  bucked  this  trend.  Here  we  have  ever-larger 
product  lines,  ever-smaller  production  runs,  small 
national  markets,  and  (consequently)  ever-higher  prices 
(Callaghan,  61)  . 

It  follows,  then,  that  an  increase  in  the  market  for  a 

weapon  system  through  arms  cooperation  agreements  would  lead 

to  fewer  product  lines,  larger  production  runs,  and  lower 

prices.   This  simple  fact  was 

.  .  .  first  appreciated  within  Europe,  and  more  so  than 
in  the  United  States.  There  were  good  reasons  for  this, 
not  least  of  which  were  the  budgetary  constraints  with 
which  the  European  states  were  faced,  the  range  of 
conventional  operational  requirements  they  had  to  meet 
with  considerable  fewer  resources,  and  the  comparatively 
small  production  run  each  could  afford  on  any  one  program 
(Edmonds,  11) . 

But   this   logic  holds   true   for   any   arms   cooperation 

agreement   in  that   all  the  participating   countries  would 

experience  cost  reductions  through 

improvements  in  production  arising  from  the 
learning  process,  through  quantity  purchase  of  materials 
and  parts,  better  use  of  fixed  capital  and  steady  cash 
flows  (Edmonds,  11)  - 

Two  political  arguments  can  be  made  as  well  in  favor  of 

arms  cooperation  agreements.   First,  a  weapon  system  that  can 

be  demonstrably  shown  to  have  reduced  unit  costs  would  be  more 

likely  to  be  accepted  by  the  citizens  of  a  country,  especially 

when  the  risks  associated  with  developing  and  producing  the 
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weapon  system  could  be  shared  with  other  nations  (Edmonds,  11- 

12).     Secondly,   a   country   that   participates   in   arms 

cooperation  agreements  with  its  allies 

demonstrates  a  willingness  to  adjust  purely 
national  priorities  in  the  interest  of  alliance  cohesion 
and,  arguably,  a  willingness  to  further  the  wider 
political  objectives  of  creating  interdependencies  among 
.  .  .  allies  and  .  .  .  establishing  an  integrated  defense 
industry  (Edmonds,  12). 

A  final  advantage  to  arms  cooperation  agreements  is  that 
it  is  militarily  more  desirable  for  alliance  forces  operating 
in  conjunction  with  each  other  to  be  equipped  with 
standardized  weapons,  logistics  and  communications  systems 
(Edmonds ,  14) . 

Set  against  these  reasons  to  enter  arms  cooperation 

agreements  are  the  pressures  inherent  in  a  sovereign  state  to 

maintain  a  unique  domestic  arms  production  capability;  it 

would  be  disadvantageous  for  a  nation  to  enter  such  an 

agreement  when  it  is  that  government's  desire 

to  limit  reliance  upon  external  sources  of 

military  equipment,  to  reduce  the  burden  of  imports  on 

their  balance  of  trade,  and  to  ensure  maximum  employment 

for  workers  in  defense  and  defense- related  industries 

(Edmonds,  1) . 

As  Shaffer  notes, 

cooperation  in  armaments  is  not  an  area  of 
national  policy  isolated  from  security  policy  or  from  more 
general  domestic  and  foreign  policy  concerns.  As  a  result 
of  this  linkage,  nations  desire  armament  self-sufficiency, 
to  the  extent  economically  and  politically  feasible,  in 
order  to  avoid  dependence  on  others  ...  in  formulating 
and  implementing  policy  objectives  (Shaffer,  27) . 


14 


In  addition,  compelling  reasons  of  national  security  could 
often  deter  one  country  from  entering  an  arms  cooperation 
agreement  due  to  the  possibility  of  sensitive  technology 
transfer. 

D.   FORCES  CONTRIBUTING  TO  ARMS  DIVERSITY 

The  earliest  attempts  at  arms  cooperation  in  NATO  led  to 
a  certain  homogeneity  of  weapons  among  the  alliance  members  in 
that  "virtually  all  military  equipment  was  supplied  by  the 
United  States"  (Shaffer,  29).  This  uniformity,  driven  by 
necessity  as  much  as  the  booming  postwar  American  economy, 
soon  fragmented. 

Five  specific  factors  have  been  identified  as  contributing 

to  the  push  toward  NATO  weapons  diversity.    The  first  of 

these  factors  is  NATO's  " overreliance  on  nuclear  weapons  to 

the   detriment   of   conventional   forces"   (Cannizzo,   57) , 

especially  in  the  early  years  of  the  NATO  alliance: 

Presuming  themselves  safe  from  conventional  attack  by 
the  USSR  and  its  allies,  (NATO)  general  purpose  forces 
were  by  and  large  starved  of  resources  and  left  to 
deteriorate;  the  member  states  were  left  free  to  a  large 
degree  to  go  their  separate,  sovereign  ways  in  pursuit  of 
national  security  ....  The  United  States  strongly 
advocated  this  reliance  on  massive  retaliation  and  did  not 
particularly  mind  the  burgeoning  proliferation  of 
different  types  of  conventional  weapons  systems  (Cannizzo, 
57)  . 

The  United  States'  support  of  a  strategic  defense  of  NATO 

should  be  viewed  in  light  of  its  then  vastly  superior  nuclear 
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arsenal  and  its  later  commitment  of  conventional  forces  to  the 

conflicts  in  Korea  and  Vietnam. 

The  second  factor  that  advanced  weapons  diversity  in  NATO 

is  closely  related  to  the  first: 

The  United  States  encouraged  the  postwar  rebuilding  of 
European  defense  industries  in  order  not  to  overtax  the 
"arsenal  of  democracy"  and  to  help  spur  reconstruction 
efforts  (Cannizzo,  57) . 

A  third   factor  derives   from  the   economic  problems 

afflicting  the  United  States  beginning  in  the  early  sixties 

brought  on  by  an  unfavorable  balance  of  trade  and  the  flow  of 

gold  passing  through  U.  S.  borders  (Cannizzo,  57): 

The  United  States  patently  became  more  interested  in 
forcing  the  Europeans  to  buy  U.  S.  equipment  and  in 
finding  ways  to  offset  the  cost  of  stationing  U.  S.  troops 
abroad  than  it  was  in  fostering  weapons  cooperation.  Such 
policies  constituted  a  source  of  irritation  for  the 
European  Allies,  which  was  compounded  by  the  "buy 
American"  acts  ....  Not  only  the  United  States  has 
continuing  economic  difficulties,  but  the  Europeans  have 
as  well,  which  today  have  helped  create  a  situation  more 
compatible  with  protectionism  than  international 
cooperation  (Cannizzo,  57) . 

Cannizzo  identifies  the  United  States'  involvement  in 

Vietnam  as   the   fourth   factor   lending   to  NATO  weapons 

diversity: 

The  diversion  of  U.  S.  interests  and  attention  to 
Southeast  Asia  clearly  reduced  U.  S.  participation  in  NATO 
affairs  and  procurement  policies  .  .  .  the  United  States 
drew  on  its  stocks  of  equipment  in  Europe  and  .  .  .  this 
left  large  gaps  which  remain  unfilled  even  today 
(Cannizzo,  57) . 

Finally,  Cannizzo  points  to  the  NATO  experience  with  arms 

cooperation  projects  themselves  as  leading  to  diversity. 

Citing  the  consortium  created  to  manage  the  coproduction  of 
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the  F-16  as  an  example,   Cannizzo  points  to  a  myriad  of 

problems : 

.  .  .  charges  of  bribery  and  even  hints  of  murder  during 
the  bargaining  and  selection  phases;  continual  cost 
increases;  acrimonious  debate  over  unfulfilled  promises  of 
jobs,  income,  and  other  benefits;  disagreements  over 
third- country  sales;  and  even  disgruntlement  over  the 
plane's  performance  (Cannizzo,  57). 

Another  example  of  an  unsuccessful  NATO  joint  program  was 
the  attempt  by  the  United  States  and  West  Germany  to  develop 
the  MBT-70  battle  tank  which  "ended  in  failure  after  more  than 
$400  million  had  been  invested"  (Kitfield,  "Finally, "  77) . 

The    current    NATO    policy    of    "Rationalization, 

Standardization,  and  Interoperability"  (RSI)  is  the  latest 

expression   of   the   alliance's   efforts   to   promote   arms 

cooperation  with  the  hope  of  realizing 

not  only  the  militarily  desirable  goal  of 
standardizing  equipment  and  harmonizing  logistical 
procedures,  but  also  the  politically  desirable  goal  of 
alliance  solidarity  (Feldman,  48) . 

E.   THE  EVOLUTION  OF  RSI 

RSI  is  intrinsically  tied  to  the  theory  of  "structural 

disarmament"  as  formulated  by  Callaghan.   Callaghan  states 

that  the  Western  nations  are 

.  .  .  experiencing  (a)  disarmament  phenomenon  of  more 
and  more  money  producing  fewer  and  fewer  weapons,  less 
"readiness,"  and  even  less  combat  sustainability .  The 
cause  is  structural .  No  nation- -not  even  the  US- -provides 
a  large  enough  defense  market  to  develop  and  produce 
everything  itself  (Callaghan,  61) . 
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This  section  will  examine  the  concept  of  RSI,  and  will  trace 
its  implementation  through  laws  enacted  by  the  United  States 
Congress  and  policy  directed  by  the  President  and  the 
Department  of  Defense. 

A  definition  of  terms  is  in  order: 

"Rationalization"  is  defined  as  any  action  which 
increases  the  effectiveness  of  NATO  by  improving  the 
command- support  structure  and  consolidating  the  various 
national  logistical  systems.  It  includes  standardization, 
specialization,  interoperability,  cross-servicing,  and 
general  cooperation  of  military  and  non-military  matters. 

"Standardization"  refers  to  the  achievement  of  the 
closest  practical  cooperation  among  forces,  sharing  of 
systems  designs,  agreement  on  operational,  administrative, 
and  logistic  procedures,  and  common  tactical  doctrine  and 
organization. 

"Interoperability"  consists  of  making  different  weapons 
systems  work  together  by  using  common  or  compatible 
components,  fuel,  and  ammunition  (Feldman,  50). 

As  was  previously  mentioned,  a  high  level  of 
standardization  was  achieved  in  the  early  years  of  NATO  by  the 
simple  fact  that  most  weapons  were  provided  by  the  United 
States.  Continued  economic  aid  to  the  European  allies  by  the 
United  States  led  to  a  rebuilding  of  European  industry  so 
that,  by  the  late  1950s,  "European  industry  was  sufficiently 
recovered  to  permit  the  independent  development  of  exclusively 
European  weapons  systems"  (Feldman,  51) . 

The  Warsaw  Pact  modernization  of  the  1970s  and  the 
realization  that  NATO  was  wasting  up  to  $15  billion  a  year 
because  of  weapons  diversity  led  to  a  renewed  call  for 
standardization  in  conjunction  with  the  rearming  of  NATO 
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(Feldman,  51) .   But  purely  national  concerns  often  overrode 

alliance  concerns  so  that 

.  .  .  NATO  leaders  found  it  increasingly  difficult  to 
convince  parliaments  to  spend  large  amounts  of  foreign 
exchange  on  the  purchase  of  weapons  from  abroad,  because 
of  their  fear  of  generating  domestic  unemployment. 
Therefore,  most  NATO  members  began  in  the  1970s  to  demand 
economic  compensation  in  the  form  of  "offsets"  such  as 
coproduction  or  countertrade  as  a  condition  of  purchase 
(Feldman,  51)  . 

In  the  United  States,  the  Government  began  to  take  the 
initiative  in  promoting  RSI.  An  amendment  to  the  Defense 
Appropriation  Authorization  Act  of  1975  called  for  the 
Secretary  of  Defense  to  ascertain  the  cost  of  the  failure  to 
standardize  weapons  in  NATO  and  to  develop  a  plan  for  arms 
cooperation  (Feldman,  51)  .  The  1976  Nunn  amendment  encouraged 
reciprocal  procurement  of  equipment  between  the  U.  S.  and 
other  NATO  nations;  authorized  the  SECDEF  to  waive  the  "Buy 
American"  law  in  the  interest  of  NATO  standardization  and 
interoperability;  and  enabled  the  establishment  of  bilateral 
reciprocal  procurement  MOUs  between  the  U.  S.  and  the  European 
nations  of  NATO  (Naval,  25) . 

President  Ford  introduced  the  two-way  street  concept  to 
the  Europeans  at  the  1976  NATO  summit,  and  advised  them  that, 
to  compete  effectively  with  U.  S.  defense  industry,  the 
Europeans  would  have  to  sacrifice  some  national  concerns  and 
combine  their  resources  (Feldman,  52) . 
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The  Carter  administration  approached  arms  cooperation  from 

two  almost  diametrically  opposing  views.  On  the  one  hand,  the 

administration 

.  .  .  wanted  to  greatly  decrease  the  U.  S.  sales  of 
weapons  around  the  world,  thus  avoiding  a  reputation  as 
"Merchants  of  Death."  Correspondence,  dubbed  the  "Leprosy 
Letter,"  was  sent  to  all  U.  S.  embassies  early  in  the 
Carter  administration  prohibiting  support  and  assistance 
to  U.  S.  defense  industry  corporations  trying  to  sell 
overseas.  Legislation  was  enacted  by  Congress  disallowing 
foreign  marketing  expenses  to  be  charged  as  a  valid  cost 
of  doing  business  (Naval,  23) . 

On  the  other  hand,  the  Carter  administration  encouraged  arms 

cooperative  agreements  with  allies;   in  particular,  Under 

Secretary  of  Defense  for  Research  and  Engineering  William 

Perry  greatly  furthered  the  cause  of  RSI.  Through  Perry's  DoD 

Directive   2010.6,   a   three-pronged  approach   to   RSI   was 

implemented: 

First,  the  general  and  reciprocal  procurement  Memoranda 
of  Understanding  (MOUs)  .  .  .  were  to  be  used  to  create  a 
"two-way  street"  in  defense  trade.  Each  country  would 
agree  to  remove  domestic  obstacles  to  the  mutual 
penetration  of  defense  markets.  Each  country  would  obtain 
exemptions  from  the  other's  customs,  duties,  tariffs,  and 
protectionist  legislation.  In  addition,  each  country 
would  ideally  give  equal  consideration  to  all  qualified 
bidders  regardless  of  the  firm's  nationality  .... 

The  second  prong  .  .  .  was  called  "Dual  Production, "  and 
attempted  to  avoid  redundant  research  and  development 
(R&D)  expenditures  by  encouraging  nations  to  produce  a 
system  already  developed  by  another  member  .... 

The  third  .  .  .  strategy  for  achieving  RSI  was  called 
the  "Family  of  Weapons"  approach  ....  it  began 
collaboration  at  the  R&D  stage,  requiring  complete  sharing 
of  technology  and  know-how.  It  called  for  the 
complementary  development  of  weapon  systems  in  a 
particular  mission  area  to  be  allocated  to  different 
countries,  sequentially  produced,  and  then  made  available 
to  other  members  for  production  or  co-production  (Feldman, 
52)  . 


20 


The  first  of  these  strategies,  the  negotiation  of  MOUs  to 
aid  in  arms  cooperation,  received  widespread  support 
throughout  NATO.  Currently,  formal  agreements  exist  between 
the  United  States  and  19  allied  countries  regarding  arms 
cooperation.  "Dual  Production"  continues  to  be  favored  by  the 
DoD  and  the  U.  S.  defense  industry  because  "it  approached 
standardization  by  relying  on  competitive  R&D  followed  by 
licensed  production  or  co-production"  (Feldman,  52).  The 
"Family  of  Weapons"  approach  was  unpopular  in  America,  on  the 
industrial  side  because  it  called  for  relinquishing 
proprietary  data  (Feldman,  52)  ,  and  on  the  military  side 
because  of  high  level  concern  "over  the  flow  of  critical 
technology  to  the  Eastern  bloc"  (Kitfield,  "Finally, "  77)  . 

The  Reagan  administration  approached  arms  cooperation  with 

an  entirely  different  attitude.   Armaments  cooperation 

.  .  .  was  pushed  on  all  fronts.  The  "Leprosy  Letter" 
was  repudiated;  the  first  international  cooperative 
development  program  was  started  (Terminally  Guided  Warhead 
for  the  Multiple  Launch  Rocket  System) ;  defense 
cooperation  offices  were  established  in  most  major  U.  S. 
embassies  to  promote  cooperation;  and  over  thirty  new 
cooperative  development  MOUs  were  signed  (Naval,  24). 

Two  other  developments  in  RSI  implementation  during  this 

period  are  the  Nunn  amendments  to  the  FY85  and  FY86  Defense 

Appropriations  Authorization  Act.   The  former  professes  the 

Government's  support  of  RSI 

.  .  .  but  intends  to  achieve  it  by  creating  an  open, 
competitive  market  in  defense  equipment- -a  trading 
structure  that  is  hoped  will  accomplish  more  than  the 
"piecemeal"  government  MOUs  of  (the)  past  (Feldman,  53). 
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The  latter  takes  a  more  proactive  role  by  requiring  all  new 

weapon   development   programs   be   examined   for   possible 

international  cooperation;  by  providing  $100M  as  "seed  money" 

for   new,   international   codevelopment   programs;   and   by 

providing  $25M  to  allow  the  U.  S.  to  test  and  evaluate  systems 

already  developed  by  Allies  prior  to  beginning  domestic 

production  of  similar  systems  (Naval,  26).    Congressional 

funding  in  support  of  cooperative  developments  climbed  from 

$100M  in  FY86,  to  $145M  in  FY87,  to  $150M  in  FY88,  to  $153M  in 

FY89,  before  falling  to  $115M  in  FY90  (Cole,  61).   Also  in 

1987,  the  Quayle  amendment  permitted  the  DoD  to  cooperate  with 

foreign  nations  from  R&D  through  procurement  (Naval,  25-26). 

President   Bush   has   generally   continued   the   arms 

cooperation   policies   of   his   predecessor.     The   Bush 

Administration  has 

.  .  .  directed  embassy  personnel  to  increase  the  level 
of  assistance  provided  to  U.  S.  defense  companies,  created 
the  Center  for  Defense  Trade  within  the  State  Department, 
and  proposed  a  "defense  GATT"  (General  Agreement  on  Trade 
and  Tariffs)  that  would  allow  free  and  open  trade  in  arms 
and  defense  technology  within  the  NATO  Alliance,  and  with 
other  U.  S.  Allies.  In  March  1991,  the  Administration 
proposed  that  the  Export- Import  Bank  guarantee  up  to  $1 
billion  in  commercial  loans  to  members  of  NATO,  Australia, 
Japan  and  Israel  to  purchase  defense  equipment  from  U.  S. 
contractors  (U.  S.  Congress,  21). 

On  April  16,  1990,  the  President  issued  his  policy  on 

offset  agreements.    In  essence,  the  policy  prohibits  the 

United  States  Government  from  participating  in 

.  .  .  any  guarantees  of  offsets  to  any  allied  government 
or  participating  industry.  This  participation  goes  so  far 
as  to  say  that  no  agency  can  encourage,  enter  directly 
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into,  or  commit  U.  S.  companies  to  any  offset  arrangements 
as  they  relate  to  any  sale  of  defense  articles  or  services 
to  other  governments.  Additionally  no  U.  S.  funds  may  be 
used  to  finance  offsets  ....  The  new  guidance  does  not 
prohibit  negotiations  for  offsets  by  any  government  with 
the  commercial  industries  involved  as  long  as  there  is  no 
U.  S.  Government  encouragement,  participation  or 
commitment  for  the  offset  agreements  which  might  be 
finally  realized  (McCarroll,  6-7). 


F.   IMPACT  OF  RSI 

Objectively,  the  impact  of  the  various  RSI  initiatives  can 
best  be  appreciated  by  examining  the  balance  in  defense  trade 
between  the  United  States  and  the  European  allies.  Most 
analysts  agree  that  in  the  1970s,  the  ratio  of  defense  trade 
between  the  U.  S.  and  its  NATO  allies  was  as  much  as  10:1  in 
favor  of  the  United  States  (Costello,  70;  Kitfield, 
"Obstacles,"  86).  Following  Perry's  RSI  initiatives,  this 
ratio  was  reduced  to  a  level  of  4.8:1  in  favor  of  the  U.  S.  in 
FY84  (Roos,  23) .  The  figures  for  FY86  show  a  further  leveling 
off  of  the  trade  ratio,  to  1.6:1  in  favor  of  the  United  States 
(Kitfield,  "Obstacles,"  86;  Roos,  23);  this  last  ratio  can  be 
interpreted  considering  Nunn  amendment.  Other  interpretations 
can  be  offered  for  the  decline- -Kitfield  points  out  that  the 
FY84  figures  are,  in  part  at  least,  because  the  European 
nations  reduced  their  defense  purchases  from  the  United  States 
by  45  percent  from  the  previous  year  (Kitfield,  "Finally, " 
76) .  However,  Congressional  calculations  show  that  the 
transatlantic  defense  trade  ratio,  which  in  1981  had  reached 
a  high  of  12:1  in  favor  of  the  U.  S.,  had  only  dropped  to 
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about  5.2:1  in  1988  (U.  S.  Congress,  49).   In  any  event,   the 

overall  trend  proves  that  some  effort  has  been  expended  in 

behalf  of  RSI. 

Another  set  of  figures  that  reinforces  the  declining  trade 

ratio  balances  concerns  military  contracts: 

The  amount  of  U.  S.  military  contracts  awarded  abroad 
nearly  doubled  (during  the  period  FY80  to  FY86)  from  $5.4 
billion  to  $9  billion  ....  During  the  same  period, 
foreign  military  sales  (FMS)  by  U.  S.  companies  were  cut 
in  half,  from  $14.8  billion  in  1980  to  $7.1  billion  (in 
1986)  (Kitfield,  "Obstacles,"  86). 

G.   OBSTACLES  ON  THE  TWO-WAY  STREET 

Although  RSI  is  vigorously  promoted  on  both  sides  of  the 
Atlantic,  very  real  obstacles  impede  optimal  progress  down  the 
"two-way  street." 

On  this  side  of  the  Atlantic,  a  general  problem  is  that 

there  is  not  a  clear,  national  consensus  in  support  of  RSI: 

RSI  never  became  a  "national  decision, "  its  proponents 
having  failed  to  win  the  blessing  of  Congress,  the 
military  services,  industry,  and  labor  ....  Each  group, 
for  a  variety  of  reasons,  has  at  one  time  or  another 
obstructed  or  resisted  RSI  (Feldman,  53). 

Four  major  obstacles  can  be  identified: 

•  the  "not  invented  here"  syndrome; 

•  economic  protectionism; 

•  technological  isolationism; 

•  the  NATO  burdensharing  controversy  (Feldman,  54) . 
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The  "not  invented  here"  syndrome  is  primarily  a  military 
criticism  of  RSI  which  rests  on  three  pillars.  First,  the 
Pentagon  is  rightfully  concerned  about  any  arms  cooperative 
program  that  could  "leave  the  U.  S.  dependent  on  a  foreign 
source  for  any  component  of  its  defense  needs"  (Feldman,  54) . 
Secondly,  the  Pentagon  reasons  that  "a  variety  of  non- 
standardized  defense  systems  would  be  more  likely  to  thwart 
Soviet  counter-measures  than  a  few  standardized  systems" 
(Feldman,  54) .  Finally,  some  officials  fear  that  the  quality 
of  our  forces'  weapons  might  be  sacrificed  to  achieve  the 
political  goal  of  armaments  cooperation  (Feldman,  54). 

The  obstacle  of  economic  protectionism  is  a  Congressional 

reaction  to  RSI.   Although  the  MOUs  were  negotiated  to  ease 

the  restrictions  of  protectionist  statutes,  elected  officials 

can  still  be  influenced  by  both  industry  and  labor  in  their 

districts.   For  instance,  the 

.  .  Aerospace  Industries  Association  (AIA)  and  the 
Electronic  Industries  Association  (EIA)  .  .  .  have 
testified  before  Congress  .  .  .  that  RSI  is  a  form  of 
"affirmative  action"  for  the  Europeans  ....  The 
implications  for  U.  S.  jobs  are  great,  according  to  the 
AIA,  which  estimates  that  for  every  $1  billion  in  export 
sales  lost,  70,000  U.  S.  jobs  disappear  as  well  (Feldman, 
55)  . 

Constraining  armaments  cooperation,  Congress  has  supported  the 

Arms  Export  Control  and  the  Export  Administration  Acts;  has 

enacted  the  requirement  that  the  Department  of  Commerce 

participate  in  the  already  slow  MOU  process;  and  has  included 
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individual  protectionist  measures  in  most  of  the  recent 
Defense  Authorization  Acts  (Naval,  26). 

Perhaps  the  greatest  obstacle  on  the  two-way  street  is  the 
issue  of  burdensharing,  which  arises  from  the  perception  that 
"some  countries  (in  the  NATO  alliance)  may  be  doing  more,  or 
not  doing  enough,  in  contributing  to  collective  security" 
(Lightburn,  26)  .  The  issue  has  often  been  couched  in  terms  of 
each  member's  financial  and  military  contribution  to  NATO 
expressed  as  a  percentage  of  GNP/GDP.  A  recently  completed 
study  by  NATO  has  explored  the  burdensharing  problem. 

The  study,  done  in  1988,  recognized  that  there  were  many 

factors,  besides  financial  and  military,  that  comprised  a 

country's  contributions  to  the  alliance,   and  that  these 

factors  could  not  all  be  quantified  or  compared  (Lightburn, 

26).   Nevertheless,  an  assessment  for  each  country's  total 

contribution  was  estimated  and  agreed  upon  by  all  other 

countries  in  the  alliance  (Lightburn,  27)  .   As  a  result  of 

these  assessments,  proposals  were  formulated  to  maintain  the 

quality  of  the  allied  defence,  to  redress  some  deficiencies 

that  the  assessments  discovered,  and  to  enhance  security  in 

the  future  (Lightburn,  27-28) : 

In  the  first  of  these  categories,  .  .  .  countries  were 
asked  to  improve  their  performance  in  meeting  NATO  goals, 
to  enhance  the  forces  which  are  committed  to  the  Alliance, 
to  improve  force  sustainability,  to  continue  to  support 
commonly  funded  programmes,  to  strive  for  improved 
armaments  cooperation,  to  provide  more  assistance  to 
developing  defence  industry  nations  and  to  search  for 
greater  efficiencies  in  the  defence  area.  In  the  second 
category,   .   .   .   countries   were   asked   to   consider 
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improvements  to  the  NATO  Composite  Force  for  North  Norway, 
to  the  Allied  Command  Europe  (ACE)  Mobile  Force  and  to  the 
on- call  naval  force  in  the  Mediterranean,  to  improve  where 
possible  host  country  support  arrangements,  to  enhance 
reinforcement  capabilities  and  to  improve  the  Alliance's 
airborne  early  warning  capability.  The  third  category 
challenged  countries  to  look  at  a  better  rationalization 
of  existing  defence  arrangements,  to  examine  and  improve 
the  potential  of  reserve  and  mobilizable  capabilities,  and 
to  explore  other  ways  of  enhancing  security  through 
additional  common  or  joint  funding  of  requirements 
(Lightburn,  28) . 


H.   DOD  SUPPORT  OF  ARMAMENTS  COOPERATIONS 

The  Department  of  Defense  has  supported  both  Executive  and 

Congressional  efforts  to  further  armaments  cooperations. 

Directives  are  in  place  which  form  the  basis  of  Defense 

Department  policy  on  international  armaments  cooperation. 

Some  key  directives  are  listed  below: 

•  DoD  Directive  2000.9,  "International  Co- Production 
Projects  and  Agreements  Between  the  United  States  and 
Other  Countries  or  International  Organizations,"  dated 
January  23,  19  74. 


DoD  Directive  3100.3,  "Cooperation  with  Allies  in  Research 
and  Development  of  Defense  Equipment, "  dated  September  27, 
1963. 


DoD  Directive  3100.4,  "Harmonization  of  Qualitative 
Requirements  for  Defense  Equipment  of  the  United  States 
and  Allies,"  dated  September  27,  1963. 


DoD  Directive  5530.3,  "International  Agreements,"  dated 
June  11,  1987. 


Directive  2000.9  is  currently  being  revised;  its  revision 
would  cancel  the  original  2000.9  and  Directives  3100.3  and 
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3100.4,  along  with  various  memoranda  issued  by  the  Department. 
The  draft  revision  proposes  a  ten-point  DoD  policy  concerning 
international  armaments  cooperation;  it  says  that  it  would  be 
DoD  policy  that 

•  The  DoD  would  cooperate  with  U.  S.  Allies  in  the  research 
and  development,  testing  and  evaluation  (RDT&E) ; 
production;  procurement;  sale  and  follow- on  support  of 
conventional  defense  equipment  when  such  cooperation 
promotes  U.  S.  foreign  policy  objectives;  U.  S.  security 
interests;  the  fulfillment  of  validated  operational 
requirements;  an  approach  for  integrating  DoD's 
requirement  generation,  acquisition  management,  and 
planning,  programming  and  budgeting  systems;  and  the 
maintenance  or  improvement  of  the  U.  S.  defense  industrial 
base. 

•  The  DoD  would  participate  in  cooperative  programs  that 
foster  the  collective  security  by  encouraging  participants 
to  adopt  interoperable  equipment,  compatible  support 
arrangements,  and  common  operational  doctrine;  by 
encouraging  participants  to  invest  appropriately  in  modern 
conventional  defense;  by  making  the  most  efficient  use  of 
the  total  scientific,  technical,  financial  and  industrial 
resources  available  to  participants;  and  by  fostering 
defense  industrial  capabilities  in  all  participating 
countries  to  meet  the  military  requirements  of  the 
alliance . 

•  The  DoD  would  maximize  its  resources  available  for  defense 
of  the  alliance  by  conserving  DoD  resources  through  access 
to  those  foreign  defense  goods  and  services  that  provide 
cost-effective  alternatives  for  meeting  U.  S.  operational 
requirements;  and  by  facilitating  the  common  defense  of 
the  alliance  through  promoting  equitable  access  to  the 
most  cost  effective  goods,  technology  and  services, 
reducing  duplicate  efforts  at  RDT&E,  production  and 
support,  and  by  fostering  a  more  efficient  market  for 
defense  goods. 

•  The  DoD  would  encourage  consensus  and  validation  of 
military  requirements  through  treaties  or  agreements  at 
the  earliest  practicable  stage  in  the  requirements' 
generation,  and  through  all  stages  of  the  requirements' 
formulation. 

•  The  DoD  would  authorize  the  release  of  classified  or 
unclassified  U.  S.  information  and  technology  if  such 
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disclosure  was  subject  to  a  Technology  Assessment  Control 
Plan  and  if  such  disclosure  complied  with  law  and  other 
DoD  Directives. 

•  The  DoD  would  protect  and  advance  the  U.  S.  industrial 
base  in  negotiating  and  implementing  cooperative 
agreements  by  considering  the  effects  of  the  agreements  on 
the  defense  industrial  base;  and  by  interfacing  with  the 
Department  of  Commerce  (DOC)  regarding  the  implications  of 
agreements  on  the  industrial  competitive  position  of  U.  S. 
industry. 

•  The  DoD  would  prepare  an  industrial  base  factors  analysis 
for  each  agreement  to  assess  the  defense  industrial 
implications  of  cooperative  agreements. 

•  The  DoD  would  not  encourage,  enter  directly  into,  or 
commit  U.  S.  firms  to  any  offset  arrangement  through  a 
cooperative  agreement  except  as  directed  by  the  President 
through  the  National  Security  Council. 

•  The  DoD  would  maintain  a  consistent  approach  to 
cooperative  agreements  by  negotiating  and  concluding  all 
agreements  in  accordance  with  DoD  Directive  5530.3  or  the 
Security  Assistance  Management  Manual,  as  applicable. 

•  The  DoD  would  continually  monitor  and  review  cooperative 
programs  to  ensure  consistency  with  U.  S.  interests  and 
objectives,  and  would  maintain  a  consolidated  index  of  all 
cooperative  agreements  (Draft,  2-6). 


DoD  Directive  553  0.3  "International  Agreements" 
establishes  the  requirements  for,  and  restrictions  on, 
authority  to  negotiate  or  conclude  an  international  agreement. 
The  directive  defines  "international  agreement"  broadly  as  any 
agreement  concluded  with  one  or  more  foreign  governments  that 
is  signed  or  agreed  to  by  personnel  of  the  U.  S.  Government, 
that  signifies  the  intention  of  the  parties  to  be  bound  in 
international  law,  and  that  is  identified  as  an  international 
agreement  or  other  similar  language  (2-1).   The  definition 
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specifically  excludes  certain  other  official  compacts  made  by 
Government  personnel  such  as  contracts  made  under  the  Federal 
Acquisition  Regulations  (FAR)  and  Foreign  Military  Sales  (FMS) 
Credit  Agreements  (2-1).  DoD  Directive  5530.3  establishes 
standard  procedures  for  requesting  authority  to  negotiate  or 
conclude  international  agreements,  and  assigns  responsibility 
for  that  authority  to  the  Under  Secretary  of  Defense  for 
Policy  (USD(P)).  For  certain  categories  of  agreements,  the 
directive  delegates  authority  to  negotiate  and  conclude 
international  agreements  to  the  Service  Secretaries,  the 
Chairman  of  the  Joint  Chiefs  of  Staff  (CJCS) ,  the  Director  of 
the  Defense  Security  Assistance  Agency  (DSAA) ,  the  Under 
Secretary  of  Defense  for  Acquisition  (USD (A)),  and  others. 

I.   DON  SUPPORT  OF  ARMAMENTS  COOPERATION 

In  December,  1989,  a  Naval  Research  Advisory  Committee 

(NRAC)  Report  on  International  Research  and  Development  (NRAC 

Report  No.  89-7)  was  completed.    The  committee  panel  was 

comprised   of   representatives   from   Government,   private 

industry,  and  universities,  and  chaired  by  Mr.  Gerald  Cann, 

then  of  General  Dynamics  Corporation.   The  panel  had  been 

formed  to  address  the  following  questions  as  they  pertained  to 

the  Navy's  international  cooperative  research  and  development 

(IR&D)  programs: 

•  How  may  the  Navy  enhance  identification  and  assessment  of 
foreign  technologies  to  facilitate  both  the  rapid  and  long 
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term  incorporation  of  these  technologies  into  naval  weapon 
systems? 

•  How  may  the  Navy  better  identify  potential  cooperative 
development  programs?  What  fundamental  elements  impact 
the  success  or  failure  of  a  cooperative  development? 
Which  elements  should  be  considered  in  determining 
priority? 

•  What  role  can  industry  assume  to  facilitate  international 
cooperative  programs  and  take  advantage  of  available 
foreign  technologies?  What  is  necessary  to  maximize 
industry  involvement  in  these  programs  and  strengthen  the 
U.  S.  industrial  base?  How  may  IR&D  funding  be  used  in 
these  programs? 

•  What  incentives  should  be  offered  to  promote  international 
cooperative  programs?  How  may  the  Navy  reward  program 
managers  and  industry  for  cost  savings  achieved  through 
successful  programs? 

•  How  may  international  programs  be  used  to  promote  the 
maritime  strategy  objectives  of  the  Navy? 

•  How  may  the  coordination  and  review  of  Navy  international 
programs  be  improved?  What  organizational  changes  are 
necessary? 

•  Are  the  fiscal  goals  established  by  the  OSD  for  service 
participation  in  international  programs,  practical  and 
adequate? 

•  How  will  new  legislation,  requiring  Department  of  Commerce 
review  of  all  proposed  international  cooperative 
development  Memoranda  of  Understanding  (MOUs) ,  impact  Navy 
programs  and  U.  S.  industry  (Naval,  15-16). 


An  immediate  cause  for  the  panel's  formation  was  the  fact 
that  two  ongoing  cooperative  programs  had  been  recently 
cancelled  and  several  others  appeared  to  be  in  trouble  (Naval, 
3).  Factors  underlying  the  cancellations  were  readily 
discernible.  These  included  changes  in  the  external 
environment  in  which  IR&D  is  conducted;  the  lack  of  a  formal 
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national  policy  on  international  armaments  cooperation;  and 

the  means  by  which  armaments  cooperative  agreements  are 

impacted  by  actions  of  Congress,  the  Department  of  Defense  and 

the  Department  of  the  Navy  (Naval,  3). 

The  panel  recognized  several  trends  in  the  external 

environment  that  impact  on  the  Navy's  IR&D  efforts,  central  of 

which  is   the   increasing   "globalization  of   the  economy, 

industry  and  trade.   This,  in  turn,  has  fostered  increasing 

interdependence  among  nations"   (Naval,   21) .    The  panel 

acknowledged  an  erosion  in  the  United  States'  traditional 

leadership  position  in  several  important 
(technological)  areas,  such  as  microelectronics,  ceramics 
and  manufacturing  process  technology.  Foreign  capability 
in  the  design  of  integrated  circuits,  launch  vehicles  and 
commercial  aircraft  underscores  the  general  trend  toward 
technology  leveling.  As  a  result,  intense  competitive 
pressure  has  promoted  widespread  dual  use  of  our  most 
advanced  technologies  (Naval,  21). 

A  final  environmental  trend  identified  by  the  panel  was 

the  fact  that  our  focus  on  the  threats  facing  our  nation  was 

shifting  from  a  military  threat  by  the  Soviet  bloc  to  an 

economic  threat  by  the  other  industrialized  nations  of  the 

world,  specifically  the  Pacific  Rim  nations  and  Western  Europe 

(Naval,  21).   The  panel  attributed  industry's  narrow 

.  .  .  preoccupation  with  short  term  profit  and  corporate 
book  value  (with  putting)  us  at  a  distinct  disadvantage  in 
dealing  with  Japan's  longer  goal  of  securing  market  share 
(Naval,  21) . 

Concerning    the   Government's    national    policy   on 

international  armaments  cooperation,  the  panel  was  quick  to 
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point  out  that  the  Government  has  no  formal  national  policy 

even  though 

.  .  .  armaments  cooperation  with  North  Atlantic  Treaty- 
Organization  (NATO)  and  other  friendly  countries  has  been 
an  integral  part  of  DoD's  RDT&E  and  production  procurement 
programs  since  World  War  II  (Naval,  23) . 

Instead  of  a  formal,  written  policy  on  international  armaments 
cooperation,  an  ad  hoc  policy  evolved  "in  accordance  with  the 
prevailing  international  views  held  by  each  administration" 
(Naval,  23)  . 

The  panel  recognized  that  the  Department  of  the  Navy  has 
generally  followed  OSD  direction  for  international  armaments 
cooperation  by  issuing  SECNAV  instructions  that  direct 
compliance  with  the  relevant  DoD  Directives  and  by  generating 
its  own  set  of  OPNAV  instructions  to  provide  guidelines  for 
conducting  MOU  negotiations  (Naval,  29).  The  NRAC  panel  did 
level  some  criticism  at  DoN  for  not  being  sufficiently 
sensitive  to  U.  S.  industrial  base  concerns  and  for  the  ultra- 
conservative  position  the  Navy  has  traditionally  taken 
regarding  the  licensing  of  its  equipment  for  use  and  sales 
overseas  (Naval,  30). 

J.   THE  NAVY  INTERNATIONAL  PROGRAMS  OFFICE  (IPO) 

Coordination  of  the  Navy's  international  armaments 
cooperative  efforts  is  accomplished  by  the  International 
Programs  Office  (IPO) ,  reporting  to  the  Deputy  Assistant 
Secretary  of  the  Navy  for  International  Policy  (DASN(IP)) .  In 
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IPO,  the  Director  for  International  Agreements  is  responsible 
for  oversight  of  the  acquisition  MOUs,  for  assisting  in  the 
development  and  negotiations  of  MOUs  by  the  Program  Managers, 
and  for  policy  coordination. 

The  IPO  office  traditionally,  at  regular  intervals,  issued 
a  "call"  for  Candidate  Nomination  Proposals  (CNPs)  throughout 
the  various  Program  Offices  of  the  Navy.  Offices  whose 
programs  were  fully  funded  typically  declined  to  submit  CNPs 
while  offices  facing  shortfalls  and  budget  cuts  proposed 
programs  for  MOU  consideration. 

Recent  procedures  established  by  DoD  Instruction  5000.2, 
"Defense  Acquisition  Management  Policies  and  Procedures, "  may 
lead  to  an  increase  in  both  the  quantity  and  quality  of 
candidate  programs.  The  instruction  requires  that  a  report  be 
submitted  to  Congress  annually  regarding  international 
cooperative  efforts.  The  report,  the  Cooperative  Research  and 
Development  Projects  Report,  is  prepared  by  the  Deputy  Under 
Secretary  of  Defense  for  International  Programs  (DUSD(IP)), 
and  reviews  Acquisition  Category  I  (ACAT  I)  ,  Major  Defense 
Acquisition  Programs,  activity  for  the  year  (ll-D-1-3).  The 
report  will  provide  Congress  with  two  pieces  of  information: 

•  A  description  of  status,  funding  and  schedule  of  existing 
cooperative  research  and  development  projects  for  which  a 
Memorandum  of  Understanding  (or  other  formal  agreement) 
has  been  entered  into; 

•  A  description  of  the  purpose,  funding  and  schedule  of  any 
proposed  new  projects  included  in  the  President's  budget 
proposal  for  which  a  Memorandum  of  Understanding  (or  other 
formal  agreement)  has  not  been  entered  into  (ll-D-1-3). 
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The  instruction  also  requires  that  a  Cooperative  Opportunities 
Document  be  included  as  Annex  G  to  the  Integrated  Program 
Summary  document  as  part  of  the  Defense  Acquisition  Board 
(DAB)  Milestones  Review  procedures  (ll-C-3)  .  This  requirement 
applies  to  all  Acquisition  Categories  programs  at  Milestones 
I  through  IV;  the  Cooperative  Opportunities  Document  would 
address  the  following  areas: 

•  Are  there  any  similar  projects  in  development  or 
production  by  one  or  more  major  allies  of  the  United 
States? 

•  If  yes,  could  that  project  satisfy,  or  be  modified  in 
scope  so  as  to  satisfy,  the  military  requirements  of  the 
United  States? 

•  What  are  the  advantages  and  disadvantages  of  seeking  to 
structure  a  cooperative  development  program  with  one  or 
more  Allied  nations? 

•  What  alternate  forms  of  cooperation  could  be  appropriate 
for  this  project  (DoD  Manual  5000. 2M,  4-H-l-l  and  -2)1 

The  IP  Office  also  targets  likely  MOU  prospects  through  the 
annual  Planning,   Programming  and  Budgeting  System  (PPBS) 
review  process,  during  which  the  office  focuses  primarily  on 
mid- level  programs  (ACAT  II  programs,  the  lower  range  of  ACAT 
I  programs,  and  the  upper  range  of  ACAT  III  programs) . 

Once  the  IP  Office  identifies  a  candidate  cooperative 
program,  either  through  a  successful  PPBS  targeting  or  a  call 
for  candidate  programs,  the  office  discusses  the  program  with 
the  originator  to  assess  the  program's  potential  as  a 
cooperative   agreement.     For   programs   with   significant 
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potential,  the  originator  drafts  and  submits  a  Project 
Nomination  Proposal  (PNP)  to  the  IPO.  The  IPO  then  performs 
the  "quick  look"  evaluation  that  is  the  subject  of  this 
research;  those  proposals  evaluated  likely  to  succeed  are 
forwarded  for  notification  to  the  Deputy  Assistant  Secretary 
of  the  Navy  for  International  Policy  (DASN(IP)  )  .  The  DASN(IP) 
either  sends  the  PNP  back  to  IPO  with  feedback  or  submits  the 
PNP  to  the  Assistant  Secretary  of  the  Navy  for  Research, 
Development  and  Acquisition  (ASN(RD&A))  for  approval  to 
Request  Authority  to  Negotiate  (RAN)  .  Once  ASN(RDStA)  approval 
is  obtained,  the  IPO  submits  the  PNP  to  the  USD  (A)  for 
authority  to  negotiate  in  accordance  with  DoD  Directive 
5530.3. 
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III.   PRESENTATION  OF  DATA 

A.   BACKGROUND  TO  THE  DEVELOPMENT  OF  SUCCESS  CRITERIA 
1.   Original  Evaluation  Procedure 

The  IP  Office  originally  used  the  following  20-point 
checklist  to  evaluate  Project  Nomination  Proposals: 

•  MOU  REQUIREMENT.  Why  must  a  MOU  be  employed?  Is  use  of 
a  MOU  based  on  legal  or  policy  grounds?  Would  use  of 
another  international  program  mechanism  (e.g.,  FMS  LOA, 
export  license,  etc.)  achieve  the  same  objectives? 

•  LEGAL.  If  a  MOU  is  required,  which  statute (s)  provide 
authority  for  the  MOU?  More  specifically,  what  is  the 
statutory  authority  for  each  of  the  obligations  undertaken 
by  the  U.  S.  Government  in  the  MOU? 

•  NUMBER  OF  PARTICIPANTS.  How  many  nations  desire  to 
participate  in  the  MOU?  (Note  that  in  general,  bilateral 
agreements  are  encouraged.) 

•  MOU  SCHEDULE.  Can  authority  to  negotiate  be  obtained  and 
can  a  MOU  be  negotiated  and  concluded  in  time  to  meet 
desired  program  schedule (s) ? 

•  PROGRAM  OBJECTIVE.  Is  the  objective  of  the  MOU  clearly 
stated?  What  is  the  motivation  and  benefit  to  the  U.  S. 
Government?  What  is  the  perceived  motivation  and  benefit 
to  the  foreign  government (s) ?  (Note  that  the  combination 
of  the  two  must  create  a  "win-win"  situation  in  order  for 
the  program  to  succeed.) 

•  PROGRAM  REQUIREMENT.  Is  there  a  documented  U.  S. 
requirement  which  has: 

-  OPNAV  support  and/or  approval? 

-  CINC/JCS  support  and/or  approval? 

-  SYSCOM/Lab  support  and/or  approval? 

-  RD&A  support  and/or  approval? 

-  OSD  support  and/or  approval? 

-  Congressional  support  and/or  approval? 
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•  Is  the  combined  U.  S. -foreign  requirement  clearly  stated 
in  the  MOU,  and  does  it  realistically  reflect  the  needs  of 
all  parties? 

•  PROGRAM  SCOPE  OF  WORK.  Are  proposed  program  tasks  and 
associated  schedules  clearly  stated?  Will  they  achieve 
program  objectives  and  meet  program  requirements  in  the 
areas  of  technical  performance,  cost  and  schedule? 

•  MANAGEMENT.  Will  this  be  a  U.  S.  managed  project,  a 
foreign  country  managed  project,  or  a  jointly  managed 
project?  What  is  the  management  hierarchy  (steering 
groups,  program  offices,  etc.)?  Will  the  program 
office  (s)  be  integrated  or  not? 

•  FINANCIAL.    What  is  the  total  cost  of  the  project 

(financial  and  non-f inancial)  ?   What  is  the  U.  S.  share 

(financial/non- financial)  versus  the  foreign  share?  What 

is   the   United   States/foreign   cost   share   rationale 

(production  off -take,  equal  share,  etc.),  and  are  the 

contributions  versus  benefits  equitable?   What  are  the 

U.  S.  sources  of  funds/non- financial  contributions?   If 

DoN  funds,  are  they  in  the  budget/POM?   If  Nunn  funds, 

have  they  been  approved  by  DoN/OSD?   If  NRC  waiver (s)  are 

proposed,   have  they  been  approved  by  DoN/OSD  and/or 

foreign  governments  involved? 

•  CURRENCY.  Will  the  program  be  managed  in  U.  S.  dollars, 
foreign  currency,  or  a  mix?  Will  the  establishment  of  a 
joint  program  bank  account  be  required? 

•  CONTRACTUAL  ARRANGEMENTS.  Will  U.  S.  contractual 
procedures  be  used  or  not?  Who  will  serve  as  the  U.  S. 
contracting  agency  (if  applicable)?  What  type  of  contract 
(FFP,  CPIF,  etc.)  will  be  employed?  Are  there  any  special 
arrangements  or  procedures?  If  so,  are  they  in  accordance 
with  the  FAR? 

•  WORK- SHARE.  If  a  cooperative  MOU,  what  is  the  anticipated 
division  of  work- share  among  participants,  by  volume  and 
by  type  (high  tech,  low  tech)  ?  Does  the  proposed  work- 
share  align  with  cost-share  (i.  e.  is  it  equitable)? 

•  OFFSET  AGREEMENTS .  Are  there  industrial  offset  agreements 
addressed  in  the  MOU  that  are  associated  with  this 
program?  Will  the  U.  S.  Government  play  a  role?  Is  that 
role  consistent  with  DoD  policy  on  offsets? 

•  INDUSTRIAL  BASE  IMPACT.  What  will  be  the  impact  on  the 
U.  S.  industrial  base  (including  sub- tier  suppliers)  of 
proposed  work- share/of f set  agreements? 
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•  INTELLECTUAL  PROPERTY  RIGHTS.  What  background  and 
foreground  data  rights  are  anticipated  to  be  transferred 
to  foreign  participants?  Do  U.  S.  rights  align  with  U.  S. 
contributions?  (Note  that  for  substantial  U.  S. 
contributions,  we  should  obtain  a  greater  degree  of  data 
rights  than  our  prospective  partners.) 

•  TECHNOLOGY  TRANSFER.  Are  sensitive  technologies  to  be 
shared  or  transferred?  Have  such  transfers  been  cleared 
through  DoN  and  OSD  (DTSA) ?  Is  TTSARB  approval  required? 
Has  disclosure  guidance  to  the  negotiating  team  been 
developed  and  issued?  Has  a  Delegation  of  Disclosure 
Letter  (DDL)  to  the  Program  Manager  (PM)  been  issued? 

•  DISCLOSURE  OF  CLASSIFIED  INFORMATION.  Will  U.  S. 
classified  information  or  material  be  disclosed  to  foreign 
nationals  during  the  course  of  the  program?  Is  approval 
by  the  National  Disclosure  Policy  Committee  (NDPC) 
required?  (Note  that  both  technology  transfer  and 
disclosure  of  CMI  should  be  addressed  in  the  MOU 
negotiating  guidance  and  DDL.) 

•  THIRD  PARTY  SALES  AND  TRANSFERS.  Will  future  U.  S. 
disclosures  require  concurrence  of  other  participants? 

(Note  that  if  the  U.  S.  possesses  considerable  "background 
equity"  in  the  program,  permitting  foreign  "veto"  of 
potential  U.  S.  sales  should  be  avoided.) 

•  MOU  CLARITY.  Does  the  MOU  clearly  state  the  arrangements 
listed  above  or  are  they  to  be  documented  by  side 
agreements? 

•  REQUIRED  DOCUMENTATION.  Have  the  following  supporting 
documents  been  prepared:  Legal  Memorandum;  Fiscal 
Memorandum,  Technology  Security  Risk  Assessment  (or  its 
successor,  Technology  Assessment  and  Control  Plan) ; 
Industrial  Base  Factors  Analysis? 


The  IP  Office  halted  use  of  the  checklist  since  it  offered  an 
entirely  subjective  overview  and  since  it  could  not  provide  a 
methodology  for  comparing  one  PNP  to  another. 
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2.   OSD  Decision  Matrix 

The  IP  Office  considered  adopting  the  following  four-part 
decision  matrix  used  by  the  Office  of  the  Secretary  of  Defense 
for  selecting  programs  to  pursue  among  those  nominated  for 
international  research  and  development  projects: 
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Though  the  OSD  evaluation  format  did  allow  for  a 
quantitative  comparison  of  one  candidate  program  to  another, 
the  IP  Office  decided  not  to  adopt  the  OSD  format  because  it 
did  not  adequately  consider  the  point  of  view  of  the  foreign 
participants  in  the  MOU.  Their  experience  with  MOUs  convinced 
the  IP  Office  that  the  key  to  the  success  of  an  MOU  was  the 
"degree  of  convergence"  between  the  U.  S.  decision  factors  and 
the  foreign  decision  factors;  a  high  degree  of  convergence 
indicated  a  synergism  at  work  in  the  MOU  that  helped  ensure  a 
successful  program. 

B.   INITIAL  SUCCESS  CRITERIA  EVALUATION 
1.   U.  S. /Foreign  Evaluation  Criteria 

On  October  5,  1990,  the  IP  Office  developed  the  following 
weighted  criteria  by  which  a  candidate  program  could  be 
evaluated: 
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II.  PROGRAMMATIC 
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2.   Five-step  Evaluation  Method 

Using  the  above  criteria,  the  IP  office  conducted  the 
following  five- step  evaluation  process  on  each  Project 
Nomination  Proposal: 

•  Establish  initial  U.  S.  "Strength  Rating"  based  on 
evaluation  of  proposed  program  vis-a-vis  IP  "program 
success"  indicators. 

•  Analyze  Foreign  Intentions/Capabilities  in  a  similar 
manner. 

•  Compare  results  of  the  first  two  steps  to  "match  up"  U. 
S. /Foreign  strengths/weaknesses.  Since  U.  S.  and  foreign 
factors  can  either  support,  neutralize,  or  conflict  with 
each  other,  the  initial  U.  S.  "Strength  Rating"  may  be 
positively  or  negatively  affected  by  the  degree  of 
convergence  between  the  U.  S.  and  Foreign  intentions  and 
capabilities . 

•  Qualitatively  determine  the  proposed  program's 
"Probability  of  Success"  and  "Payoff  Potential"  if  program 
is  successful. 

•  Develop  a  consolidated  priority  listing  which  takes  into 
account  the  program's  quantitative  "Composite  Strength 
Rating, "  and  qualitative  "Probability  of  Success"  and 
"Payoff  Potential"  assessments. 


The  individual  criteria  for  each  area  (Level  of 
Commitment,  Programmatic  Factors,  Ability  to  Contribute)  were 
assigned  values  from  one  to  ten;  an  average  for  each  area  was 
then  determined  and  multiplied  by  a  predetermined  weight 
(0.75,  1.5,  and  1.0  for  Level  of  Commitment,  Programmatic 
Factors,  and  Ability  to  Contribute,  respectively)  to  arrive  at 
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an  area  "Strength  Rating."  Area  strength  ratings  for  both  the 
United  States  and  the  foreign  participants  were  compared, 
summed  and  multiplied  by  a  second  weight  to  arrive  at  a 
composite  area  strength  rating.   This  second  weight  (ranging 
in  multiples  of  0.25  from  0.50  to  1.50)  was  subjectively- 
determined  by  the  IP  Office  by  aligning  the  individual 
criteria   under   each   U.   S.   area   with   their   foreign 
counterparts.  If  a  positive  degree  of  convergence  existed  for 
an  area,  the  weight  would  be  a  value  greater  than  one.   A 
value  of  one  was  assigned  for  a  neutral  degree  of  convergence, 
and  a  value  less  than  one  was  assigned  for  a  negative  degree 
of  convergence.   These  composite  area  strength  ratings  were 
then  summed  to  determine  the  final  composite  value  by  which 
programs  would  be  ranked. 

3.   Initial  Application  Of  Evaluation  Criteria  and  Method 
a.  Programs  Evaluated 

The  following  programs  were  initially  evaluated;  the  short 
titles  in  parentheses  are  for  cross-referencing  purposes: 
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PROGRAM  TITLE 

FOREIGN  PARTICIPANTS 

NATO  MECHANICAL 
MINESWEEPING  SYSTEM 
(NMMS) 

FRANCE,  NETHERLANDS, 
SPAIN,  ITALY 

ANTI-SHIP  MISSILE 
COUNTERMEASURES 
(ASMCM) 

UNITED  KINGDOM 

"SPINNAKER"  (SPIN) 

CANADA 

U.  S.  PACOM  COMBINED 
INTEROPERABILITY  PROGRAM 
(CIP) 

AUSTRALIA,  SOUTH  KOREA, 
JAPAN 

NATO  CSNI  (CSNI) 

UNITED  KINGDOM,  FRANCE, 

NETHERLANDS ,  CANADA 

CONTINUOUS  PROCESSING  OF 
SOLID  PROPELLANTS 
(C2P2) 

FRANCE 

b.    Evaluation  Sheets 

The  individual  evaluation  sheets  for  the  above  programs 
are  recreated  on  the  following  six  pages.  Although  the  format 
of  these  recreations  differs  slightly  from  the  originals  due 
to  software  limitations,  the  data  and  general  sense  of  the 
evaluations  remain  true  to  the  originals. 
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PROGRAM  SHORT  TITLE;  NMMS 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  3 

-Service  8 

-Strength  Rating  3.00 


FOREIGN 


Level  of  Commitment  (0.75) 


Legislature 
MOD 

Service/Equiv. 
■Strength  Rating 


1 
6 
9 
4.00 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective  4 

-Military  Reqm' ts  10 

-Alliance  Reqm' ts  10 

-MOU/ Program  5 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  5 

Objectives 

-Strength  Rating  9.75 


Programmatic  Factors  (1.5 


-Program  Objective 
-Military  Reqm' ts 
-Alliance  Reqm' ts 
-MOU/ Program 

Schedules 
-Industrial  Base 

Impact 
-3rd  Party  Sales 

Objectives 
-Strength  Rating 


Degree  of  Convergence  Weight:   1.25 
Ability  to  Contribute  (1.0)       Ability  to  Contribute 


-Funding 

10 

-Government 

10 

Technology 

-  Industry 

8 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

9.50 

4 
10 
10 

5 

5 
5 
9.75 

(1.0 


-  Funding 

10 

-Government 

10 

Technology 

- Industry 

8 

Technology 

-Technology 

10 

Transfer 

-Security 

5 

-Strength  Rating 

8.60 

Degree  of  Convergence  Weight:   1.5 
U.  S.  Strength      22.25  Foreign  Strength 

Probability  of  Success:   Medium-High 
Payoff  Potential:   Medium-High 


22.35 


COMPOSITE  STRENGTH 


58.53 
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PROGRAM  SHORT  TITLE:  ASMCM 


UNITED  STATES 

FOREIGN 

Level  of  Commitment 

(0.75) 

Level  of  Commitment 

(0.75) 

-Congress 

-OSD 

-Service 

-Strength  Rating 

1 
1 
6 
2.00 

-Legislature 

-MOD 

- Service/Equiv . 

-Strength  Rating 

1 
3 
5 
2.25 

Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective  7 

-Military  Reqm'ts  10 

-Alliance  Reqm'ts  NA 

-MOU/ Program  10 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  12.00 


Programmatic  Factors  (1.5 


-Program  Objective 

7 

-Military  Reqm'ts 

5 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

10.13 

Degree  of  Convergence  Weight:   1.25 


Ability  to  Contribute  (1.0) 


-Funding 

7 

-Government 

10 

Technology 

-Industry 

NA 

Technology 

-Technology 

5 

Transfer 

-Security 

NA 

-Strength  Rating 

7.33 

Ability  to  Contribute  (1.0) 


-  Funding 

7 

-Government 

10 

Technology 

-  Industry 

NA 

Technology 

-Technology 

5 

Transfer 

-Security 

10 

-Strength  Rating 

8.00 

Degree  of  Convergence  Weight:   1.5 
U.  S.  Strength      21.33  Foreign  Strength 

Probability  of  Success:   High-High 
Payoff  Potential:   High-High 
COMPOSITE  STRENGTH 54.91 


20.38 
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PROGRAM  SHORT  TITLE;  SPIN 


UNITED  STATES 

FOREIGN 

Level  of  Commitment 

(0.75) 

Level  of  Commitment 

(0.75) 

-Congress 

-OSD 

-Service 

-Strength  Rating 

1 
3 
8 
3.00 

-Legislature 

-MOD 

- Service/Equiv . 

-Strength  Rating 

1 
5 
8 
3.50 

Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5 

-Program  Objective  6 
-Military  Reqm' ts  10 
-Alliance  Reqm'ts  NA 
-MOU/ Program        9 

Schedules 
-Industrial  Base 

Impact 
-3rd  Party  Sales 

Objectives 
-Strength  Rating 


5 
NA 

11.25 
Degree  of  Convergence  Weight:   1.2  5 


Programmatic  Factors  (1.5) 

-Program  Objective  6 

-Military  Reqm'ts  10 

-Alliance  Reqm'ts  NA 

-MOU/ Program  9 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  11.25 


Ability  to  Contribute  (1.0) 


-Funding 

10 

-Government 

8 

Technology 

- Industry 

5 

Technology 

-Technology 

7 

Transfer 

-Security 

NA 

-Strength  Rating 

7.50 

Ability  to  Contribute  (1.0) 


-Funding 

10 

-Government 

8 

Technology 

-Industry 

5 

Technology 

-Technology 

7 

Transfer 

-Security 

10 

-Strength  Rating 

8.00 

Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      21.75  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   High-High 

COMPOSITE  STRENGTH      54.00 


22.75 
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PROGRAM  SHORT  TITLE:  CIP 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  8 

-Service  7 

-Strength  Rating  4.00 


FOREIGN 
Level  of  Commitment  (0.75) 


-Legislature 
-MOD 

-Service/Equiv. 
-Strength  Rating 


1 
8 
9 
4.50 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm'ts  8 

-Alliance  Reqm'ts  NA 

-MOU/Program  10 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  10.50 


Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm'ts  8 

-Alliance  Reqm'ts  NA 

-MOU/Program  10 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  10.50 


Degree  of  Convergence  Weight:   1.25 
Ability  to  Contribute  (1.0)       Ability  to  Contribute  (1.0) 


-Funding 

5 

-Government 

8 

Technology 

-  Industry 

NA 

Technology 

-Technology 

8 

Transfer 

-Security 

NA 

-Strength  Rating 

7.00 

-Funding 

9 

-Government 

8 

Technology 

- Industry 

NA 

Technology 

-Technology 

8 

Transfer 

-Security 

10 

-Strength  Rating 

8.75 

Degree  of  Convergence  Weight:   1.0 

U.  S.  Strength      21.50  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   Medium-High 

COMPOSITE  STRENGTH 50.50 


23.75 
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PROGRAM  SHORT  TITLE;  CSNI 


UNITED  STATES 

FOREIGN 

Level  of  Commitment 

(0.75) 

Level  of  Commitment 

(0.75) 

-Congress 

-OSD 

-Service 

-Strength  Rating 

3 
4 
6 
3.25 

-Legislature 

-MOD 

- Service/Equiv . 

-Strength  Rating 

2 

5 
6 
3.25 

Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5)         Programmatic  Factors  (1.5) 

-Program  Objective  1 

-Military  Reqm'ts  2 

-Alliance  Reqm'ts  10 

-MOU/ Prog ram  8 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  7.80 

Degree  of  Convergence  Weight:   1.25 

Ability  to  Contribute  (1.0)       Ability  to  Contribute  (1.0) 


-Program  Objective 

3 

-Military  Reqm'ts 

3 

-Alliance  Reqm'ts 

10 

-MOU/ Program 

8 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

8.70 

-  Funding 

5 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

8.33 

-Funding 

10 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

5 

-Strength  Rating 

8.75 

Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      19.38  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   Medium-Medium 

COMPOSITE  STRENGTH 48.48 


20.70 
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PROGRAM  SHORT  TITLE:  C2P2 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  2 

-Service  4 

-Strength  Rating  1.75 


FOREIGN 
Level  of  Commitment  (0.75) 


■Legislature 
•MOD 

Service/Equiv. 
■Strength  Rating 


1 
4 
6 
2.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective   1 
-Military  Reqm'ts    1 

-Alliance  Reqm'ts  NA 
-MOU/ Program        6 

Schedules 

-Industrial  Base  10 

Impact 

-3rd  Party  Sales  NA 

Objectives 
-Strength  Rating     6.75 


Programmatic  Factors  (1.5) 


-Program  Objective 

1 

-Military  Reqm'ts 

1 

-Alliance  Reqm'ts 

NA 

-MOU/ Prog ram 

6 

Schedules 

-Industrial  Base 

10 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

6.75 

Degree  of  Convergence  Weight:   1.0 


Ability  to  Contribute  (1.0) 


-  Funding 

2 

-Government 

10 

Technology 

- Industry 

7 

Technology 

-Technology 

8 

Transfer 

-Security 

NA 

-Strength  Rating 

6.75 

Ability  to  Contribute  (1.0) 


-  Funding 

8 

-Government 

10 

Technology 

- Industry 

7 

Technology 

-Technology 

8 

Transfer 

-Security 

5 

-Strength  Rating 

7.60 

Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      15.25  Foreign  Strength 

Probability  of  Success:   Medium-Medium 

Payoff  Potential:   Medium-High 

COMPOSITE  STRENGTH 35.94 


17.10 
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c.    Results  of  Initial  Application 

The  following  table  summarizes  the  results  of  this  initial 
evaluation: 


PROGRAM 
NAME 

COMPOSITE 

STRENGTH 

RATING 

U.  S. 

STRENGTH 

RATING 

FOREIGN 

STRENGTH 

RATING 

PROBABIL- 
ITY OF 
SUCCESS 

PAYOFF 
POTEN- 
TIAL 

NMMS 

58.53 

22.25 

22.35 

MEDIUM- 
HIGH 

HIGH 

ASMCM 

54.91 

21.33 

20.38 

HIGH 

HIGH 

SPIN 

54.00 

21.75 

22  .75 

MEDIUM- 
HIGH 

HIGH 

CIP 

50.50 

21.50 

23.75 

MEDIUM- 
HIGH 

MEDIUM- 
HIGH 

CSNI 

48.48 

19.38 

20.70 

MEDIUM- 
HIGH 

MEDIUM 

C2P2 

35.94 

15.25 

17.10 

MEDIUM 

MEDIUM- 
HIGH 

4.   Second  Application  of  Evaluation  Criteria  and  Method 
a.  Programs  Evaluated 

An  additional  12  programs  were  evaluated  later  in  the  same 
fiscal  year.   These  programs  are  listed  below: 
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PROGRAM  TITLE 

FOREIGN  PARTICIPANTS 

REMOTE  SENSING  UNIT  WITH 
SYNTHETIC  APERTURE  RADAR 
(RSU) 

NORWAY , 

UNITED  KINGDOM 

AIRBORNE  ELECTROMAGNETIC 
SENSOR  (AES) 

CANADA 

TACTICAL  LIGHTWEIGHT 
SURVEILLANCE  SATELLITE 
(TLSS) 

AUSTRALIA 

TACTICAL  TARGET  SUPPORT 
(TTS) 

AUSTRALIA 

DISTRIBUTED  TACTICAL 
SURVEILLANCE  (DTS) 

UNITED  KINGDOM 

TACTICAL  SHIP 
SURVEILLANCE  (TSS) 

UNITED  KINGDOM 

OCEANOGRAPHY  COOPERATION 
(OC-J) 

JAPAN 

OCEANOGRAPHY  COOPERATION 
(OC-N) 

NORWAY 

OCEAN  COLOR  ASSESSMENT, 
G-I-N  SEA  (OCA-G) 

UNITED  KINGDOM,  NORWAY, 

ICELAND,  DENMARK 

OCEAN  COLOR  ASSESSMENT, 
MEDITERRANEAN  SEA 
(OCA-M) 

UNITED  KINGDOM,  NORWAY, 
ICELAND,  DENMARK 

MILLIMETER  WAVE  SEA 
SURVEILLANCE  (MWSS) 

GERMANY,  UNITED  KINGDOM 

REMOTE  SPECTROMETRY 
RESEARCH  (RSR) 

CANADA,  UNITED  KINGDOM 

b.    Evaluation  Sheets 

The  following  pages  replicate  the  evaluation  sheets 
resulting  from  this  second  application  of  the  success  criteria 
and  5 -step  evaluation  method  to  candidate  programs: 
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PROGRAM  SHORT  TITLE;   RSU 


UNITED  STATES 

FOREIGN 

Level  of  Commitment 

(0.75) 

Level  of  Commitment 

(0.75) 

-Congress 

-OSD 

-Service 

-Strength  Rating 

1 
1 
8 
2.50 

-Legislature 

-MOD 

- Service/Equiv . 

-Strength  Rating 

1 
1 
8 
2.50 

Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 


Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm'ts  10 

-Alliance  Reqm'ts  5 

-MOU/ Program  7 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  9.60 

Degree  of  Convergence  Weight:   1.25 

Ability  to  Contribute  (1.0)       Ability  to  Contribute 


-Program  Objective 

5 

-Military  Reqm'ts 

10 

-Alliance  Reqm'ts 

5 

-MOU/ Program 

7 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

9.60 

-Funding 

7 

-Government 

9 

Technology 

- Industry 

10 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

9.00 

(1.0) 


-Funding 

5 

-Government 

9 

Technology 

- Industry 

10 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

8.80 

Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      21.10  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   Medium-High 

COMPOSITE  STRENGTH      51.2  5 


20.90 
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PROGRAM  SHORT  TITLE;  AES 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  4 

-Strength  Rating  1.50 


FOREIGN 


Level  of  Commitment  (0.75) 


-Legislature 

-MOD 

- Service/Equiv . 

-Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5 

-Program  Objective  3 
-Military  Reqm'ts  3 
-Alliance  Reqm'ts  NA 
-MOU/ Program       10 

Schedules 
-Industrial  Base 

Impact 
-3rd  Party  Sales 

Objectives 
-Strength  Rating 


Programmatic  Factors  (1.5) 


4 
NA 
7.50 
Degree  of  Convergence  Weight 


-Program  Objective 

3 

-Military  Reqm'ts 

3 

-Alliance  Reqm'ts 

NA 

-MOU/Program 

10 

Schedules 

-Industrial  Base 

4 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

7.50 

1.25 


Ability  to  Contribute  (1.0 


Ability  to  Contribute  (1.0) 


-Funding 
-Government 

Technology 
-  Industry 

Technology 
-Technology 

Transfer 
-Security 
-Strength  Rating 


7 

10 

10 

10 

NA 
9.25 


-Funding 
-Government 

Technology 
- Industry 

Technology 
-Technology 

Transfer 
-Security 
-Strength  Rating 


Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      18.25  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   Medium 

COMPOSITE  STRENGTH 45.31 


7 
10 

10 

10 

10 
9.40 


18.65 
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PROGRAM  SHORT  TITLE;  TLSS 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 


Level  of  Commitment  (0.75) 


-Legislature 

-MOD 

-  Service /Equiv. 

-Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective  7 
-Military  Reqm'ts  1 
-Alliance  Reqm'ts  7 
-MOU/ Program       10 

Schedules 
-Industrial  Base 

Impact 
-3rd  Party  Sales 

Objectives 
-Strength  Rating 


Programmatic  Factors  (1.5) 


7 
NA 
9.60 
Degree  of  Convergence  Weight:   1.25 


-Program  Objective 

7 

-Military  Reqm'ts 

1 

-Alliance  Reqm'ts 

7 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

4 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

8.70 

Ability  to  Contribute  (1.0) 


-  Funding 

2 

-Government 

10 

Technology 

-Industry 

10 

Technology 

-Technology 

5 

Transfer 

-Security 

NA 

-Strength  Rating 

6.75 

Ability  to  Contribute  (1.0) 


-  Funding 

2 

-Government 

10 

Technology 

- Industry 

4 

Technology 

-Technology 

5 

Transfer 

-Security 

10 

-Strength  Rating 

6.20 

Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      18.10  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   Medium-High 

COMPOSITE  STRENGTH 42.56 


16.65 
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PROGRAM  SHORT  TITLE;  TTS 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 


Level  of  Commitment  (0.75) 


•Legislature 

MOD 

Service/Equiv. 
•Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 


Programmatic  Factors  (1.5) 


-Program  Objective 

5 

-Program  Objective 

5 

-Military  Reqm' ts 

2 

-Military  Reqm'ts 

2 

-Alliance  Reqm' ts 

7 

-Alliance  Reqm'ts 

7 

-MOU/ Program 

8 

-MOU/ Program 

8 

Schedules 

Schedules 

-Industrial  Base 

6 

-Industrial  Base 

6 

Impact 

Impact 

-3rd  Party  Sales 

NA 

-3rd  Party  Sales 

NA 

Objectives 

Objectives 

-Strength  Rating 

8.40 

-Strength  Rating 

8. 

40 

Degree 

of  Converg* 

Bnce  Weight:   1.25 

Ability  to  Contribut 

:e  (1.0) 

Ability  to  Contribute 

(1. 

0) 

-Funding 

1 

-Funding 

1 

-Government 

10 

-Government 

10 

Technology 

Technology 

- Industry 

10 

-  Industry 

10 

Technology 

Technology 

-Technology 

5 

-Technology 

5 

Transfer 

Transfer 

-Security 

NA 

-Security 

10 

-Strength  Rating 

6.50 

-Strength  Rating 

7. 

20 

Degree  of  Convergence  Weight:   1.25 

U.  S.  Strength      16.65  Foreign  Strength 

Probability  of  Success:   Medium-High 

Payoff  Potential:   Medium 

COMPOSITE  STRENGTH 41.63 


17.35 
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PROGRAM  SHORT  TITLE;  DTS 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 
Level  of  Commitment  (0.75) 


-Legislature 
-MOD 

-Service/Equiv. 
-Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.51 

-Program  Objective   2 

-Military  Reqm'ts  10 

-Alliance  Reqm'ts  NA 
-MOU/ Program        7 

Schedules 
-Industrial  Base     5 

Impact 

-3rd  Party  Sales  NA 

Objectives 
-Strength  Rating     9.00 


Programmatic  Factors  (1.5) 


-Program  Objective 

2 

-Military  Reqm'ts 

10 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

7 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

9.00 

Degree  of  Convergence  Weight:   1.00 


Ability  to  Contribute  (1.0 


-Funding 

5 

-Government 

8 

Technology 

- Industry 

5 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

7.00 

Ability  to  Contribute  (1.0) 


-Funding 

5 

-Government 

8 

Technology 

- Industry 

5 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

7.60 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      17.75  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium 

COMPOSITE  STRENGTH 36.10 


18.35 
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PROGRAM  SHORT  TITLE;  TSS 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 


Level  of  Commitment  (0.75) 


-Legislature 
-MOD 

-Service/Equiv. 
-Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5)         Programmatic  Factors  (1.5) 

-Program  Objective  1 

-Military  Reqm'ts  10 

-Alliance  Reqm'ts  NA 

-MOU/ Program  8 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  9.00 

Degree  of  Convergence  Weight:   1.00 

Ability  to  Contribute  (1.0)       Ability  to  Contribute 


-Program  Objective 

1 

-Military  Reqm'ts 

10 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

8 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

9.00 

-Funding 

6 

-Government 

10 

Technology 

-Industry 

5 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

7.75 

1.0 


-  Funding 

4 

-Government 

10 

Technology 

-Industry 

5 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

7.80 

Degree  of  Convergence  Weight:   0.75 

U.  S.  Strength      18.50  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium-Low 

COMPOSITE  STRENGTH 33.16 


18.55 
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PROGRAM  SHORT  TITLE:  PC -J 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 
Level  of  Commitment  (0.75) 


-Legislature 
-MOD 

-Service/Equiv. 
-Strength  Rating 


1 
1 
1 
0.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.51 

-Program  Objective   2 
-Military  Reqm'ts    3 

-Alliance  Reqm'ts  NA 

-MOU/ Program  10 

Schedules 
-Industrial  Base     5 

Impact 

-3rd  Party  Sales  NA 

Objectives 
-Strength  Rating     7.50 


Programmatic  Factors  (1.5) 


-Program  Objective 

2 

-Military  Reqm'ts 

3 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

7.50 

Degree  of  Convergence  Weight:   1.00 


Ability  to  Contribute  (1.0) 


-Funding 

1 

-Government 

10 

Technology 

-  Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

7.00 

Ability  to  Contribute  (1.0 


-Funding 

1 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

7.75 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      16.25  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium 

COMPOSITE  STRENGTH 32.25 


16.00 
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PROGRAM  SHORT  TITLE:  OC-N 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 
Level  of  Commitment  (0.75) 


-Legislature 
-MOD 

-Service/Equiv. 
-Strength  Rating 


1 
1 
1 
0.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 


Programmatic  Factors  (1.5) 

-Program  Objective  2 

-Military  Reqm'ts  3 

-Alliance  Reqm'ts  NA 

-MOU/ Program  10 

Schedules 

-Industrial  Base  5 

Impact 

-3rd  Party  Sales  NA 

Objectives 

-Strength  Rating  7.50 

Degree  of  Convergence  Weight:   1.00 

Ability  to  Contribute  (1.0)        Ability  to  Contribute 


-Program  Objective 

2 

-Military  Reqm'ts 

3 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

7.50 

-Funding 

1 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

7.00 

1.0) 


-Funding 

1 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

7.75 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      16.25  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium 

COMPOSITE  STRENGTH 32.2  5 


16.00 
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PROGRAM  SHORT  TITLE:  OCA-G 


UNITED  STATES 

FOREIGN 

Level  of  Commitment 

(0.75) 

Level  of  Commitment 

(0.75) 

-Congress 

-OSD 

-Service 

-Strength  Rating 

1 
1 
4 
1.50 

-Legislature 

-MOD 

- Service /Equiv. 

-Strength  Rating 

1 
1 
1 
0.75 

Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective  1 
-Military  Reqm'ts  2 
-Alliance  Reqm'ts  2 
-MOU/ Program       10 

Schedules 
-Industrial  Base 

Impact 
-3rd  Party  Sales 

Objectives 
-Strength  Rating 


Programmatic  Factors  (1.5) 


5 
NA 
6.00 
Degree  of  Convergence  Weight:   1.00 


-Program  Objective 

1 

-Military  Reqm'ts 

2 

-Alliance  Reqm'ts 

2 

-MOU/ Prog ram 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

6.00 

Ability  to  Contribute  (1.0) 


-Funding 

6 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

8.67 

Ability  to  Contribute  (1.0) 


-Funding 

5 

-Government 

10 

Technology 

- Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

8.75 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      16.17  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium-Low 

COMPOSITE  STRENGTH 31.67 


15.50 
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PROGRAM  SHORT  TITLE:  OCA-M 


UNITED  STATES 

Level  of  Commitment  (0.75! 

-Congress  1 

-OSD  1 

-Service  4 

-Strength  Rating  1.50 


FOREIGN 


Level  of  Commitment  (0.75) 


-Legislature 
-MOD 

-Service/Equiv. 
-Strength  Rating 


1 
1 
1 
0.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 

-Program  Objective  1 
-Military  Reqm'ts  2 
-Alliance  Reqm'ts  2 
-MOU/ Program       10 

Schedules 
-Industrial  Base 

Impact 
-3rd  Party  Sales 

Objectives 
-Strength  Rating 


Programmatic  Factors  (1.5 


5 
NA 
6.00 
Degree  of  Convergence  Weight:   1.00 


-Program  Objective 

1 

-Military  Reqm'ts 

2 

-Alliance  Reqm'ts 

2 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

6.00 

Ability  to  Contribute  (1.0) 


-Funding 

6 

-Government 

10 

Technology 

-Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

NA 

-Strength  Rating 

8.67 

Ability  to  Contribute  (1.0) 


-Funding 

5 

-Government 

10 

Technology 

-Industry 

NA 

Technology 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

8.75 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      16.17  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium-Low 

COMPOSITE  STRENGTH 31.67 


15.50 
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PROGRAM  SHORT  TITLE;  MWSS 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 
Level  of  Commitment  (0.75) 


■Legislature 
■MOD 

■Service/Equiv. 
•Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5) 


Programmatic  Factors  (1.5) 

-Program  Objective   1 
-Military  Reqm'ts    4 

-Alliance  Reqm'ts  NA 

-MOU/ Program  10 

Schedules 
-Industrial  Base     8 

Impact 

-3rd  Party  Sales  NA 

Objectives 
-Strength  Rating     8.63 

Degree  of  Convergence  Weight:   1.00 

Ability  to  Contribute  (1.0)       Ability  to  Contribute 


-  Program  Ob j  ect  ive 

1 

-Military  Reqm'ts 

4 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

7.50 

-  Funding 

1 

-Government 

10 

Technology 

- Industry 

8 

Technology 

-Technology 

5 

Transfer 

-Security 

NA 

-Strength  Rating 

6.00 

(1.0) 


-Funding 

1 

-Government 

10 

Technology 

- Industry 

8 

Technology 

-Technology 

5 

Transfer 

-Security 

5 

-Strength  Rating 

5.80 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      16.38  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium-Low 

COMPOSITE  STRENGTH 31.43 


15.05 
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PROGRAM  SHORT  TITLE:  RSR 


UNITED  STATES 

Level  of  Commitment  (0.75] 

-Congress  1 

-OSD  1 

-Service  5 

-Strength  Rating  1.75 


FOREIGN 


Level  of  Commitment  (0.75) 


Legislature 
•MOD 

Service/Equiv. 
Strength  Rating 


1 
1 
5 
1.75 


Degree  of  Convergence  Weight:   1.0 


Programmatic  Factors  (1.5 

-Program  Objective   1 
-Military  Reqm'ts    1 

-Alliance  Reqm'ts  NA 

-MOU/ Program  10 

Schedules 
-Industrial  Base    5 

Impact 

-3rd  Party  Sales  NA 

Objectives 
-Strength  Rating     6.38 


Programmatic  Factors  (1.5 


-Program  Objective 

1 

-Military  Reqm'ts 

1 

-Alliance  Reqm'ts 

NA 

-MOU/ Program 

10 

Schedules 

-Industrial  Base 

5 

Impact 

-3rd  Party  Sales 

NA 

Objectives 

-Strength  Rating 

6.38 

Degree  of  Convergence  Weight:   1.00 


Ability  to  Contribute  (1.0 


-Funding 

1 

-Government 

8 

Technology 

-Industry 

5 

Technology 

-Technology 

5 

Transfer 

-Security 

NA 

-Strength  Rating 

4.75 

Ability  to  Contribute  (1.0) 


-Funding 

2 

-Government 

8 

Technology 

- Industry 

5 

Technology 

-Technology 

5 

Transfer 

-Security 

10 

-Strength  Rating 

6.00 

Degree  of  Convergence  Weight:   1.00 

U.  S.  Strength      12.88  Foreign  Strength 

Probability  of  Success:   Medium 

Payoff  Potential:   Medium 

COMPOSITE  STRENGTH 27.00 


14.13 
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c.    Results  of  Second  Application 

This  second  set  of  program  evaluations  is  summarized  in 
the  table  below: 


PROGRAM 
NAME 

COMPOSITE 

STRENGTH 

RATING 

U.  S. 

STRENGTH 

RATING 

FOREIGN 

STRENGTH 

RATING 

PROBABIL- 
ITY OF 
SUCCESS 

PAYOFF 
POTEN- 
TIAL 

RSU 

51.25 

21.10 

20.90 

MEDIUM- 
HIGH 

MEDIUM- 
HIGH 

AES 

45.31 

18.25 

18.65 

MEDIUM- 
HIGH 

MEDIUM 

TLSS 

42.56 

18.10 

16.65 

MEDIUM- 
HIGH 

MEDIUM- 
HIGH 

TTS 

41.63 

16.65 

17.35 

MEDIUM- 
HIGH 

MEDIUM 

DTS 

36.10 

17.75 

18.35 

MEDIUM 

MEDIUM 

TSS 

33.16 

18.50 

18.55 

MEDIUM 

MEDIUM- 
LOW 

OC-J 

32.25 

16.25 

16.00 

MEDIUM 

MEDIUM 

OC-N 

32.25 

16.25 

16.00 

MEDIUM 

MEDIUM 

OCA-G 

31.67 

16.17 

15.50 

MEDIUM 

MEDIUM- 
LOW 

OCA-M 

31.67 

16.17 

15.50 

MEDIUM 

MEDIUM- 
LOW 

MWSS 

31.43 

16.38 

15.05 

MEDIUM 

MEDIUM- 
LOW 

RSR 

27.00 

12.88 

14. 13 

MEDIUM 

MEDIUM- 
LOW 
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C.   REVISED  SUCCESS  CRITERIA  EVALUATION 

1.   Revised  U.  S. /Foreign  Evaluation  Criteria 

On  July  30,  1991,  the  IP  office  revised  the  evaluation 
criteria.  The  revision  separated  two  criteria  ("Technology 
Transfer"  and  "Security")  from  the  "Ability  to  Contribute" 
area  and  established  them  in  a  new  area,  "Technology/Security 
Factors."  Under  the  "Programmatic  Factors"  area,  the  revision 
deleted  one  criterion  ("Alliance  Requirements"),  replaced  two 
criteria  ("Industrial  Base  Impact"  and  "3rd  Party  Sales 
Objectives")  with  new  ones  ("Workshare"  and  "Payoff"),  and 
added  NATO  operational  requirements  criteria  to  the  "Military 
Requirements"  section.  The  revised  worksheet  is  provided 
below: 


I.  LEVEL  OF 

COMMITMENT: 

10 

5 

1 

-CONGRESS/ 

EQUIVALENT 

LEGISLATION 

MEMBER/STAFF 

NO  INTEREST 

-OSD/MOD 

USD  (A)  UP 

DUSD/ASD  UP 

WORKING 
LEVEL 

-SERVICE/ 
EQUIVALENT 

ASN(RDA)  UP 

SPONSOR 
SYSCOM/PEO 

WORKING 
LEVEL 
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II.   PROGRAMMATIC 

10 

5 

1 

FACTORS 

-  PROGRAM 

MAJOR  SYSTEM 

OTHER  SYSTEM 

TECHBASE/ 

OBJECTIVE 

FLEET 

FLEET 

STUDIES 

DELIVERY 

DELIVERY 

-MILITARY 

APPROVED 

APPROVED 

INTENT/NO 

REQUIREMENTS 

OR/NSR 

TOR/NST 

OR  REQUIRED 

-MOU/ PROGRAM 

NO  MOU/ 

CONSTRAINED 

MAJOR 

SCHEDULES 

PROGRAM 

MOU/ PROGRAM 

SCHEDULE 

SCHEDULE 

SCHEDULE 

DISCONNECT 

PROBLEMS 

-WORKSHARE 

WORKSHARE 

WORKSHARE 

WORKSHARE 

EQUALS 

W/IN  PLUS  OR 

DOES  NOT 

COSTSHARE 

MINUS  10%  OF 

EQUAL 

COSTSHARE 

COSTSHARE 

-  PAYOFF 

GREATLY 

ENHANCES 

REDUNDANT 

ENHANCES 

FLEET 

SYSTEM 

FLEET 

EFFECTIVE- 

EFFECTIVE- 

NESS 

NESS 

III.   ABILITY  TO 

CONTRIBUTE 

10 

5 

1 

-FUNDING 

FULLY  FUNDED 
IN  POM 

PARTIALLY 
FUNDED  IN 
POM 

INTENT  TO 
POM 

-GOVERNMENT 
TECHNOLOGY 

FULL  SYSCOM/ 
LAB  SUPPORT 

PARTIAL 
SYS COM/ LAB 

SUPPORT 

SYS COM/ LAB 

RESOURCE 

PROBLEMS 

IV.   TECHNOLOGY/ 
SECURITY 
FACTORS 

10 

5 

1 

-TECHNOLOGY 
TRANSFER 

NO  TECH 

TRANSFER 
ISSUES 

SELECTED 
TECH  TRANS- 
FER ISSUES 

MAJOR  TECH 

TRANSFER 

ISSUES 

-SECURITY 

LOW  RISK 

MEDIUM  RISK 

HIGH  RISK 
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2 .  Revised  Evaluation  Method 

The  evaluation  process  associated  with  this  new  set  of 
criteria  differs  from  the  initial  process.  The  weights  (0.75, 
1.50,  and  1.00)  of  the  initial  process  are  again  assigned  to 
the  areas  of  "Level  of  Commitment,"  "Programmatic  Factors," 
and  "Ability  to  Contribute,"  and  a  weight  of  1.00  is  assigned 
to  the  new  area  "Technology/Security  Factors."  U.  S.  and 
foreign  strength  ratings  are  computed  as  before,  but  the 
composite  strength  rating  computation  differs.  Under  the 
initial  process,  each  area's  strength  ratings  (U.  S.  and 
foreign)  were  first  summed  and  then  multiplied  by  subjectively 
assigned  weights;  the  resultant  products  were  then  summed  to 
arrive  at  the  composite  strength  rating.  Under  the  revised 
procedure,  all  area  strength  ratings  (both  U.  S.  and  foreign) 
are  summed  and  multiplied  by  a  single,  subjectively  assigned 
weight  factor.  Summary  U.  S.  and  foreign  strength  ratings  are 
not  computed  under  the  revised  process;  neither  are 
qualitative  measures  given  to  "Probability  of  Success"  or 
"Payoff  Potential." 

3.  Application  of  Revised  Evaluation  Criteria  and  Method 
a.    Programs  Evaluated 

Four  programs  were  evaluated  using  the  revised  criteria 
and  procedures.  The  following  table  identifies  these 
programs : 
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PROGRAM  TITLE 

FOREIGN  PARTICIPANTS 

MARITIME  UNMANNED  AERIAL 
VEHICLE  (MUAV) 

CANADA,  FRANCE,  GERMANY, 
NETHERLANDS ,  NORWAY , 
UNITED  KINGDOM 

ADVANCED  STEEL  SYSTEMS 
(ADV  ST) 

JAPAN 

ALUMINUM  OXYGEN  BATTERIES 
(AL  BAT) 

CANADA 

REMOTE  ATMOSPHERIC  AND 
IONOSPHERIC  DETECTION 
SYSTEM  (RAIDS) 

AUSTRALIA,  BELGIUM, 
CANADA,  GERMANY,  SWEDEN 

b.    Evaluation  Sheets 

The  evaluation  sheets  representing  these  four  programs  are 
reproduced  on  the  following  pages: 
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PROGRAM  SHORT  TITLE:  MUAV 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  6 

-Strength  Rating  2.00 

Programmatic  Factors  (1.5) 

-Program  Objective  8 

-Military  Reqm'ts  8 

-MOU/ Program  10 

Schedules 

-Workshare  5 

-Payoff  8 

-Strength  Rating  11.70 

Ability  to  Contribute  (1.0) 


-Funding 

10 

-Government 

0 

Technology 

-Strength  Rating 

10. 

00 

Technoloqy/ Security 

(1. 

0) 

-Technology 

10 

Transfer 

-Security 

10 

-Strength  Rating 

10. 

,00 

FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  0 

-MOD  1 

-Service/Equiv.  6 

-Strength  Rating  2.63 

Programmatic  Factors  (1.5) 

-Program  Objective  8 

-Military  Reqm'ts  7 

-MOU/Program  10 

Schedules 

-Workshare  5 

-Payoff  8 

-Strength  Rating  11.40 

Ability  to  Contribute  (1.0) 

-Funding  5 

-Government  0 

Technology 

-Strength  Rating  5.00 

Technology/Security  (1.0) 

-Technology  10 

Transfer 

-Security  0 

-Strength  Rating  10.00 


OVERALL  STRENGTH  MATCHING   1.00 


COMPOSITE  STRENGTH   62.73 
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PROGRAM  SHORT  TITLE;  ADV  ST 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  3 

-Strength  Rating  1.75 

Programmatic  Factors  (1.5) 

-Program  Objective  6 

-Military  Reqm' ts  3 

-MOU/ Prog ram  2 

Schedules 

-Workshare  0 

-Payoff  8 

-Strength  Rating  7.13 

Ability  to  Contribute  (1.0) 

-Funding  5 

-Government  10 

Technology 

-Strength  Rating  7.50 

Technology/Security  (1.0) 

-Technology  9 

Transfer 

-Security  10 

-Strength  Rating  9.50 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  5 

-Service/Equiv.  0 

-Strength  Rating  2.25 

Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm' ts  0 

-MOU/Program  0 

Schedules 

-Workshare  0 

-Payoff  7 

-Strength  Rating  9.00 

Ability  to  Contribute  (1.0) 


-  Funding 
-Government 

Technology 
-Strength  Rating 


8.00 

Technology/Security  (1.0) 

9 


-Technology 

Transfer 
-Security 
-Strength  Rating 


0 
9.00 


OVERALL  STRENGTH  MATCHING   1.2  5 


COMPOSITE  STRENGTH   58.00 
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PROGRAM  SHORT  TITLE;  AL  BAT 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  3 

-Service  5 

-Strength  Rating  2.25 

Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm'ts  10 

-MOU/ Program  5 

Schedules 

-Workshare  0 

-Payoff  9 

-Strength  Rating  10.88 

Ability  to  Contribute  (1.0) 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 
-MOD  1 

-Service/Equiv.  1 
-Strength  Rating    0.75 

Programmatic  Factors  (1.5) 

-Program  Objective  0 
-Military  Reqm'ts  0 
-MOU/ Program        5 

Schedules 

-Workshare  2 

-Payoff  2 
-Strength  Rating    4.50 

Ability  to  Contribute  (1.0) 


-Funding  10 

-Government  10 

Technology 

-Strength  Rating  10.00 


-Funding  7 

-Government  0 

Technology 

-Strength  Rating  7.00 


Technology/ Security 

(1. 

_oi 

Technology /Security 

(1. 

0) 

-Technology 

Transfer 
-Security 
-Strength  Rating 

10 

0 
10. 

.00 

-Technology 

Transfer 
-Security 
-Strength  Rating 

10 

10 
10. 

.00 

OVERALL  STRENGTH  MATCHING   1.00 


COMPOSITE  STRENGTH   55.3  8 
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PROGRAM  SHORT  TITLE:  RAIDS 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  2 

-Strength  Rating  1.00 

Programmatic  Factors  (1.5) 

-Program  Objective  2 

-Military  Reqm' ts  8 

-MOU/ Prog ram  8 

Schedules 

-Workshare  0 

-Payoff  4 

-Strength  Rating  8.25 

Ability  to  Contribute  (1.0) 

-Funding  5 

-Government  10 

Technology 

-Strength  Rating  7.50 

Technology/Security  (1.0) 

-Technology  5 

Transfer 

-Security  5 

-Strength  Rating  5.00 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  1 

-Service/Equiv.  1 

-Strength  Rating  0.75 

Programmatic  Factors  (1.5) 


-Program  Objective 
-Military  Reqm'ts 
-MOU/ Program 
Schedules 

5 
0 
0 

-Workshare 

0 

-Payoff 
-Strength  Rating 

2 

5. 

25 

Ability  to  Contribute  (1.0) 


-Funding 
-Government 

Technology 
-Strength  Rating 


3 
10 


6.50 
Technology/Security  (1.0) 


-Technology 

Transfer 
-Security 
-Strength  Rating 


0 
5.00 


OVERALL  STRENGTH  MATCHING   1.00 
COMPOSITE  STRENGTH   39.25 
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c.    Re -evaluation  of  Programs  Using  Revised  Evaluation 
Criteria  and  Method 

Attempting  to  standardize  the  evaluation  process,  the  IP 
office  re-evaluated  the  six  programs  initially  evaluated 
(NMMS,  ASMCM,  SPIN,  CIP,  CSNI ,  and  C2P2)  using  the  new 
criteria  and  procedures.  These  re -evaluations  are  reproduced 
on  the  following  pages: 
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PROGRAM  SHORT  TITLE:  NMMS 


UNITED  STATES 


FOREIGN 


Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  3 

-Service  8 

-Strength  Rating  3.00 

Programmatic  Factors  (1.5) 

-Program  Objective  4 

-Military  Reqm'ts  10 

-MOU/Program  5 

Schedules 

-Workshare  5 

-Payoff  8 

-Strength  Rating  9.60 

Ability  to  Contribute  (1.0) 

-Funding  10 

-Government  10 

Technology 

-Strength  Rating  10.00 

Technology/Security  (1.0) 

-Technology  10 

Transfer 

-Security  5 

-Strength  Rating  7.50 


Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  6 

-Service/Equiv.  9 

-Strength  Rating  4.00 

Programmatic  Factors  (1.5) 

-Program  Objective  4 

-Military  Reqm'ts  10 

-MOU/Program  10 

Schedules 

-Workshare  5 

-Payoff  8 

-Strength  Rating  11.10 

Ability  to  Contribute  (1.0) 

-Funding  10 

-Government  10 

Technology 

-Strength  Rating  10.00 

Technology/Security  (1.0) 

-Technology  10 

Transfer 

-Security  0 

-Strength  Rating  10.00 


OVERALL  STRENGTH  MATCHING   1.5  0 
COMPOSITE  STRENGTH   7  5.20 
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PROGRAM  SHORT  TITLE;  ASMCM 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  1 

-Service  6 

-Strength  Rating  2.00 

Programmatic  Factors  (1.5) 

-Program  Objective  7 

-Military  Reqm' ts  10 

-MOU/ Program  10 

Schedules 

-Workshare  5 

-Payoff  10 

-Strength  Rating  12.60 

Ability  to  Contribute  (1.0) 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  3 

-Service/Equiv.  5 

-Strength  Rating  2.25 

Programmatic  Factors  (1.5) 

-Program  Objective  7 

-Military  Reqm' ts  5 

-MOU/ Program  10 

Schedules 

-Workshare  5 

-Payoff  10 

-Strength  Rating  11.10 

Ability  to  Contribute  (1.0) 


-Funding  7 

-Government  10 

Technology 

-Strength  Rating  8.50 

Technology/Security  (1.0) 

-Technology  5 

Transfer 

-Security  10 

-Strength  Rating  5.00 


-  Funding  7 

-Government  10 

Technology 

-Strength  Rating  8.50 

Technology/Security  (1.0) 

-Technology  5 

Transfer 

-Security  0 

-Strength  Rating  5.00 


OVERALL  STRENGTH  MATCHING   1.50 
COMPOSITE  STRENGTH   65.95 
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PROGRAM  SHORT  TITLE;  SPIN 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  3 

-Service  8 

-Strength  Rating  3.00 

Programmatic  Factors  (1.5) 

-Program  Objective  6 

-Military  Reqm' ts  10 

-MOU/ Program  9 

Schedules 

-Workshare  5 

-Payoff  10 

-Strength  Rating  12.00 

Ability  to  Contribute  (1.0) 

-Funding  10 
-Government         8 

Technology 
-Strength  Rating     9.00 

Technology/Security  (1.0) 

-Technology         7 

Transfer 

-Security  10 
-Strength  Rating     8.50 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  5 

-Service/Equiv.  8 

-Strength  Rating  3.50 

Programmatic  Factors  (1.5) 

-Program  Objective  6 

-Military  Reqm' ts  10 

-MOU/ Program  9 

Schedules 

-Workshare  5 

-Payoff  10 

-Strength  Rating  12.00 

Ability  to  Contribute  (1.0) 


-Funding 
-Government 

Technology 
-Strength  Rating 

10 
8 

9. 

00 

Technology /Security 

(1. 

0) 

-Technology 

Transfer 
-Security 
-Strength  Rating 

7 

0 
7. 

00 

OVERALL  STRENGTH  MATCHING   1.25 
COMPOSITE  STRENGTH   68.50 
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PROGRAM  SHORT  TITLE;  CIP 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  8 

-Service  7 

-Strength  Rating  4.00 

Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm'ts  8 

-MOU/ Program  10 

Schedules 

-Workshare  5 

-Payoff  8 

-Strength  Rating  10.80 

Ability  to  Contribute  (1.0) 

-Funding  5 

-Government  8 

Technology 

-Strength  Rating  6.50 

Technology/Security  (1.0) 

-Technology  8 

Transfer 

-Security  10 

-Strength  Rating  9.00 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  8 

-Service/Equiv.  9 

-Strength  Rating  4.50 

Programmatic  Factors  (1.5) 

-Program  Objective  5 

-Military  Reqm'ts  8 

-MOU/Program  10 

Schedules 

-Workshare  5 

-Payoff  8 

-Strength  Rating  10.80 

Ability  to  Contribute  (1.0) 

-  Funding  9 

-Government  8 

Technology 

-Strength  Rating  8.50 

Technology/Security  (1.0) 

-Technology  8 

Transfer 

-Security  0 

-Strength  Rating  8.00 


OVERALL  STRENGTH  MATCHING   1.0  0 
COMPOSITE  STRENGTH   62.10 


PROGRAM  SHORT  TITLE;  CSNI 

UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  3 

-OSD  4 

-Service  6 

-Strength  Rating  3.25 

Programmatic  Factors  (1.5) 

-Program  Objective  1 

-Military  Reqm'ts  7 

-MOU/ Program  8 

Schedules 

-Workshare  5 

-Payoff  5 

-Strength  Rating  7.80 

Ability  to  Contribute  (1.0) 


-Funding 
-Government 

Technology 
-Strength  Rating 


5 
10 


7.50 

Technology/Security  (1.0) 

10 


-Technology 

Transfer 
-Security 
-Strength  Rating 


5 
7.50 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  2 

-MOD  5 

-Service/Equiv.  6 

-Strength  Rating  3.25 

Programmatic  Factors  (1.5) 

-Program  Objective  3 

-Military  Reqm'ts  8 

-MOU/ Program  8 

Schedules 

-Workshare  5 

-Payoff  5 

-Strength  Rating  8.70 

Ability  to  Contribute  (1.0) 


Funding 
Government 

Technology 
Strength  Rating 


10 
10 


10.00 

Technology/Security  (1.0) 

10 


-Technology 

Transfer 
-Security 
-Strength  Rating 


0 
10.00 


OVERALL  STRENGTH  MATCHING   1.2  5 


COMPOSITE  STRENGTH   62.38 
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PROGRAM  SHORT  TITLE:  C2P2 


UNITED  STATES 

Level  of  Commitment  (0.75) 

-Congress  1 

-OSD  2 

-Service  4 

-Strength  Rating  1.75 

Programmatic  Factors  (1.5) 

-Program  Objective  1 

-Military  Reqm'ts  1 

-MOU/ Program  6 

Schedules 

-Workshare  10 

-Payoff  6 

-Strength  Rating  7.20 

Ability  to  Contribute  (1.0 


-  Funding 
-Government 

Technology 
-Strength  Rating 


2 

10 


6.00 

Technology/Security  (1.0) 

-Technology  8 

Transfer 

-Security  5 

-Strength  Rating  6.50 


FOREIGN 

Level  of  Commitment  (0.75) 

-Legislature  1 

-MOD  4 

-Service/Equiv.  6 

-Strength  Rating  2.75 

Programmatic  Factors  (1.5) 

-Program  Objective  1 

-Military  Reqm'ts  1 

-MOU/Program  6 

Schedules 

-Workshare  10 

-Payoff  6 

-Strength  Rating  7.20 

Ability  to  Contribute  (1.0) 


-Funding 
-Government 

Technology 
-Strength  Rating 

Technology /Security 

8 
10 

9. 

(1. 

00 
0) 

-Technology 

Transfer 
-Security 
-Strength  Rating 

8 

0 
8. 

00 

OVERALL  STRENGTH  MATCHING   1.25 
COMPOSITE  STRENGTH   52.15 
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d.    Results    of    Application    of    Revised    Criteria    and 
Method 

The  results  of  the  evaluation  conducted  using  the  revised 
criteria  and  method  are  summarized  in  the  table  below, 
compared,  when  applicable,  to  the  results  of  the  initial 
evaluation: 


PROGRAM 

INITIAL 
EVALUATION 

INITIAL 
RANK 

REVISED 
EVALUATION 

REVISED 
RANK 

NMMS 

58.53 

1 

75.20 

1 

SPIN 

54.00 

3 

68.50 

2 

ASMCM 

54.91 

2 

65.95 

3 

MUAV 

NA 

NA 

62.73 

4 

CSNI 

48.48 

5 

62.38 

5 

CIP 

50.50 

4 

62.10 

6 

ADV  ST 

NA 

NA 

58.00 

7 

AL  BAT 

NA 

NA 

55.38 

8 

C2P2 

35.94 

6 

52.15 

9 

RAIDS 

NA 

NA 

39.25 

10 

D.   ANOMALIES  RESULTING  FROM  APPLICATION  OF  REVISED  CRITERIA 
AND  METHOD 

Several  anomalies  are  readily  apparent  regarding  the  two 
evaluation  procedures.  Of  primary  significance  is  the  fact 
that  the  four  programs  being  evaluated  for  the  first  time 
generally  scored  lower  than  those  programs  being  re- evaluated 
using  the  revised  procedures.   Additionally,  for  two  pairs 
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(ASMCM/SPIN  and  CIP/CSNI) ,  the  revised  rankings  are  the 
reverse  of  the  initial  rankings.  These  anomalies  caused  the 
IP  office  to  question  whether  there  might  be  problems  with  the 
revised  procedure;  evaluations  using  the  revised  procedure 
were  halted  until  a  review  could  be  conducted. 
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IV:   ANALYSIS  OF  DATA 

A.   SPREADSHEET  ANALYSIS  OF  INITIAL  EVALUATION  CRITERIA  AND 
METHOD 
1.   Construction  of  Spreadsheet 

To  analyze  the  IPO  evaluation  process  required  translating 
the  data  from  the  various  sets  of  evaluation  sheets  into  a 
spreadsheet  program  that  replicated  both  the  appearance  and 
the  functions  of  the  evaluation  sheet.  The  cell  contents  of 
an  example  of  this  spreadsheet  program  are  given  below: 


Al 

E3 

A4 

E4 

A6 

E6 

A8 

C8 

E8 

G8 

A9 

C9 

E9 

G9 

A10 

CIO 

E10 

G10 

All 

Bll 

Ell 

Fll 

A12 

C12 

E12 

G12 

A14 


NATO  MECHANICAL  MINESWEEPING  SYSTEM 

FOREIGN:   FRANCE,  SPAIN, 

UNITED  STATES 

THE  NETHERLANDS,  ITALY 

LEVEL  OF  COMMITMENT 

LEVEL  OF  COMMITMENT 

CONGRESS 

1 

LEGISLATURE 

1 

OSD 

3 

MOD 

6 

SERVICE 

8 

SERVICE/EQUIV. 

9 

WEIGHT 

0.75 

WEIGHT 

0.75 

STRENGTH  RATING 

@SUM(C8. .CIO) /@COUNT(C8. .CIO) *B11 

STRENGTH  RATING 

@SUM(G8. .G10) /@COUNT(G8. .G10) *F11 

PROGRAMMATIC  FACTORS 


86 


E14 

:   PROGRAMMATIC  FACTORS 

A16 

:   PROGRAM  OBJECTIVES 

C16 

4 

E16 

:   PROGRAM  OBJECTIVES 

G16 

4 

A17 

:   MILITARY  RQMTS 

C17 

.   10 

E17 

:   MILITARY  RQMTS 

G17 

:   10 

A18 

ALLIANCE  RQMTS 

C18 

10 

E18 

ALLIANCE  RQMTS 

G18 

10 

A19 

MOU/PROGRAM  SKED 

C19 

5 

E19 

MOU/PROGRAM  SKED 

G19 

5 

A2  0. 

INDUST  BASE  IMPACT 

C2  0. 

5 

E20. 

INDUST  BASE  IMPACT 

G2  0 

5 

A21. 

THIRD  PARTY  SALES 

C21: 

5 

E21: 

THIRD  PARTY  SALES 

G21: 

5 

A22: 

WEIGHT 

B22: 

1.5 

E22: 

WEIGHT 

F22: 

1.5 

A2  3. 

STRENGTH  RATING 

C23: 

@SUM(C16. .C21) /@COUNT(C16. 

.C21) *B22 

E23: 

STRENGTH  RATING 

G23: 

@SUM(G16. .G21) /©COUNT (G16. 

.G21) *F22 

A2  5 

ABILITY  TO  CONTRIBUTE 

E25 

ABILITY  TO  CONTRIBUTE 

A2  7 

FUNDING 

C27 

10 

E27 

FUNDING 

G2  7 

:   10 

A2  8 

GOVT  TECHNOLOGY 

C2  8 

:   10 

E28 

:   GOVT  TECHNOLOGY 

G2  8 

:   10 

A2  9 

:   INDUST  TECHNOLOGY 

C29 

:   8 

E29 

:   INDUST  TECHNOLOGY 

G29 

:   8 

A3  0 

:   TECH  TRANSFER 

C30 

:   10 

E30 

:   TECH  TRANSFER 

G30 

:   10 
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A31 
C31 
E31 
G31 
A3  2 
B32 
E32 
F32 
A3  3 
C33 
E33 
G33 
A3  5 
C35 
E35 
G35 
D38 
D40 
H4  0 
D41 
F41 
H41 
A4  3 
D43 
F4  3 
H43 
A44 
D44 
F44 
H44 
A4  5 
D4  5 
F4  5 
H4  5 
E47 
H4  7 


SECURITY 

SECURITY 

5 

WEIGHT 

1 

WEIGHT 

1 

STRENGTH  RATING 

@SUM(C27. .C31)/@COUNT(C2  7. .C31) *B32 

STRENGTH  RATING 

@SUM(G27. .G31) /@C0UNT(G27. .G31) *F32 

TOTAL  STRENGTH 

+C12+C23+C33 

TOTAL  STRENGTH 

+G12+G23+G33 

DEGREE  OF  CONVERGENCE 

COMBINED 

COMPOSITE 

STRENGTH 

WEIGHT 

WEIGHT 

LEVEL  OF  COMMITMENT 

©VALUE (CI 2 ) +OVALUE (G12 ) 

1 

+D43*F43 

PROGRAMMATIC  FACTORS 

@VALUE ( C2  3 ) +@VALUE ( G2  3 ) 

1.25 

+D44*F44 

ABILITY  TO  CONTRIBUTE 

©VALUE ( C3  3 ) +@VALUE ( G3  3 ) 

1.5 

+D4  5*F4  5 

TOTAL  COMPOSITE  STRENGTH 

+H43+H44+H45 


While  creating  the  spreadsheet,  it  was  discovered  that, 
for  any  line  item  on  the  evaluation  sheet  not  given  a 
numerical  score  (i.e.  "Security"  under  the  United  States' 
"Ability  to  Contribute"),  the  corresponding  cell  on  the 
spreadsheet  (C31)  had  to  be  completely  blank.  Any  entry  (i. 
e.  NA,  0,  or  even  a  global  justification  command)  in  such  a 
cell  would  be  counted  as  an  entry  in  the  subsequent  "@COUNT" 
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formula  used  to  compute  the  area  strength  rating  (in  this 
instance  cell  C33)  and  would  result  in  an  incorrect 
calculation. 

2.   Results  of  Spreadsheet  Analysis 

The  key  formula  cells  of  this  spreadsheet  (C12,  G12 ,  C23, 
G23,  C33,  G33,  C35,  G35,  and  H47)  duplicated  the  values  of  the 
area  strength  ratings,  the  total  strength  ratings,  and  the 
composite  strength  rating  of  the  evaluation  sheet.  The  values 
represented  by  the  other  formula  cells  (D43,  H43,  D44,  H44, 
D45,  H45)  do  not  appear  on  the  evaluation  sheet  but  were 
included  in  the  spreadsheet  for  ease  of  computation. 

This  spreadsheet  analysis  was  applied  to  the  evaluation 
sheets  representing  the  initial  evaluation  performed  on  the 
following  candidate  programs: 

•  NATO  Mechanical  Minesweeping  System 

•  Anti-Ship  Missile  Countermeasure 

•  "Spinnaker" 

•  U.  S.  PACOM  Combined  Interoperability  Program 

•  NATO  CSNI 

•  Continuous  Processing  of  Solid  Propellants 

•  Remote  Sensing  Unit  with  Synthetic  Aperture  Radar 

•  Airborne  Electromagnetic  Sensor 

•  Tactical  Lightweight  Surveillance  Satellite 

•  Tactical  Target  Support 

•  Distributed  Tactical  Surveillance 
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•  Tactical  Ship  Surveillance 

•  Oceanography  Cooperation- -Japan 

•  Oceanography  Cooperation- -Norway 

•  Ocean  Color  Assessment  (G-I-N  Sea) 

•  Ocean  Color  Assessment  (Mediterranean  Sea) 

•  Millimeter  Wave  Sea  Surveillance 

•  Remote  Spectrometry  Research. 

In  every  instance,  the  spreadsheet  analysis  of  these  programs 
directly  correlated  to  all  values  on  the  evaluation  sheets. 

B.   SPREADSHEET  ANALYSIS  OF  REVISED  EVALUATION  CRITERIA  AND 
METHOD 
1.   Construction  of  Spreadsheet 

A  second,  similar  spreadsheet  based  on  the  evaluation 
sheets  of  the  revised  evaluation  criteria  and  procedures  was 
then  created;  the  criteria  scores  and  weights  of  the  Maritime 
Unmanned  Aerial  Vehicle  (MUAV)  evaluation  sheet  were  used  in 
creating  the  spreadsheet.  The  following  represents  the  cell 
contents  of  this  second  spreadsheet: 


Al 
E2 
E3 
A4 
E4 
A6 
E6 
A8 
C8 
E8 


MUAV 

FOREIGN:   CANADA,  FRANCE 

GERMANY,  NETHERLANDS 

UNITED  STATES 

NORWAY,  UNITED  KINGDOM 

LEVEL  OF  COMMITMENT 

LEVEL  OF  COMMITMENT 

CONGRESS 

1 

LEGISLATURE 
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G8: 

A9: 

OSD 

C9: 

1 

E9: 

MOD 

G9: 

1 

A10: 

SERVICE 

CIO: 

6 

ElO: 

SERVICE/EQUIV. 

GIO: 

6 

All: 

WEIGHT 

Bll: 

0.75 

Ell: 

WEIGHT 

Fll: 

0.75 

A12: 

STRENGTH  RATING 

C12: 

@SUM(C8. .CIO) /@COUNT(C8. .CIO) *B11 

E12: 

STRENGTH  RATING 

G12: 

@SUM(G8. .GIO) /@COUNT(G8. .GIO) *F11 

A14: 

PROGRAMMATIC  FACTORS 

E14: 

PROGRAMMATIC  FACTORS 

A16: 

PROGRAM  OBJECTIVES 

C16: 

8 

E16: 

PROGRAM  OBJECTIVES 

G16: 

8 

A17: 

MILITARY  RQMTS 

C17: 

8 

E17: 

MILITARY  RQMTS 

G17: 

7 

A18: 

MOU/ PROGRAM  SKED 

CI  8: 

10 

E18: 

MOU/ PROGRAM  SKED 

G18: 

10 

A19: 

WORKSHARE 

C19: 

5 

E19: 

WORKSHARE 

G19: 

5 

A2  0: 

PAYOFF 

C2  0: 

8 

E20: 

PAYOFF 

G20: 

8 

A21: 

WEIGHT 

B21: 

1.5 

E21: 

WEIGHT 

F21: 

1.5 

A22: 

STRENGTH  RATING 

C22: 

@SUM(C16. .C20) /@COUNT(C16. .C20) *B21 

E22: 

STRENGTH  RATING 

G22: 

@SUM(G16. .G20) /@COUNT(G16. .G20) *F21 

A24: 

ABILITY  TO  CONTRIBUTE 

E24: 

ABILITY  TO  CONTRIBUTE 

A2  6: 

FUNDING 

C2  6: 

10 
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E26 
G2  6 
A2  7 
C2  7 
E27 
G27 
A2  8 
B28 
E28 
F28 
A2  9 
C2  9 
E29 
G2  9 
A31 
E31 
A33 
C33 
E33 
G33 
A3  4 
C34 
E34 
G34 
A35 
B35 
E35 
F35 
A3  6 
C36 
E36 
G36 
A4  0 
D40 
A41 
041 
A4  3 
D4  3 


FUNDING 

5 

GOVT  TECHNOLOGY 

GOVT  TECHNOLOGY 

WEIGHT 

1 

WEIGHT 

1 

STRENGTH  RATING 

@SUM(C26. -C27) /0COUNT (C26. .C27) *B28 

STRENGTH  RATING 

@SUM(G26. .G27) /@COUNT (G26. .G27) *F28 

TECHNOLOGY / SECURI T Y 

TECHNOLOGY/SECURITY 

TECH  TRANSFER 

10 

TECH  TRANSFER 

10 

SECURITY 

10 

SECURITY 

WEIGHT 

1 

WEIGHT 

1 

STRENGTH  RATING 

@SUM(C33. .C34) /0COUNT (C33. .C34) *B35 

STRENGTH  RATING 

@SUM(G33. -G34) /@COUNT (G33. -G34) *F35 

TOTAL  COMBINED  STRENGTH 

+C12+G12+C22+G22+C2  9+G2  9+C3  6+G3  6 

STRENGTH  MATCHING 

1 

TOTAL  COMPOSITE  STRENGTH 

+D40-D41 


2 .   Results  of  Spreadsheet  Analysis 

With  this  spreadsheet,  the  formula  cells  corresponding  to 
the  area  strength  ratings  (C12,  G12,  C22,  G22,  C29,  G29,  C36 
and  G36)  produced  values  identical  with  the  values  on  the 
evaluation  sheet.   The  composite  strength  formula  cell  (D43) , 
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however,  produced  a  value  different  from  the  evaluation  sheet 
(97.80  vice  75.20) .  This  second  spreadsheet  was  applied  to 
all  the  programs  that  had  undergone  the  revised  evaluation 
procedure,  using  the  criteria  scores  and  weights  from  the 
evaluation  sheets.  Always,  the  results  were  the  same:  area 
strength  ratings  corresponded  to  the  evaluation  sheets  but 
composite  strength  ratings  differed.  The  following  table 
summarizes  the  results  of  this  step  in  the  analysis: 


PROGRAM 

SHORT 

TITLE 

SPREAD- 
SHEET 
COMPOSITE 
STRENGTH 
RATING 

SPREAD- 
SHEET 
RANKING 

EVALUA- 
TION 
SHEET 
COMPOSITE 
STRENGTH 
RATING 

EVALUA- 
TION 
SHEET 
RANKING 

NMMS 

97.80 

1 

75.20 

1 

ASMCM 

86.18 

2 

65.95 

3 

SPIN 

80.00 

3 

68.50 

2 

CSNI 

72.50 

4 

62.38 

5 

ADV  ST 

67.  66 

5 

58.00 

7 

MUAV 

62.73 

6 

62.73 

4 

CIP 

62.10 

7 

62.10 

6 

C2P2 

60.50 

8 

52.15 

9 

AL  BAT 

55.38 

9 

55.38 

8 

RAIDS 

39.25 

10 

39.25 

10 
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Since  the  spreadsheet  calculations  corresponded  to  the 
evaluation  sheets  in  all  area  strength  ratings,  and  differed 
in  all  composite  strength  ratings,  the  cell  governing  the 
computation  of  "Composite  Strength"  was  examined  further. 

C.   REASON  FOR  DISCREPANCY  BETWEEN  EVALUATION  SHEETS  AND 
SPREADSHEET  ANALYSIS 

The  "Composite  Strength"  value  on  the  evaluation  sheet  was 
being  calculated  in  a  manner  different  from  the  intent  of  the 
evaluation  sheet.  Referring  back  to  the  MUAV  spreadsheet 
analysis,  "Composite  Strength"  is  calculated  by  the  formula 
"+D40*D41"  or,  substituting  in  for  cell  D40,  by  the  formula 

"+ (C12+G12+C22+G22+C29+G29+C36+G36) *D41. " 

This  formula  captures  the  intent  of  the  revised  procedure:  to 
apply  a  single  "Strength  Matching"  weight  (D41)  to  the  sum  of 
all  the  area  strength  ratings. 

The  IPO  evaluation  sheets,  however,  calculate  "Composite 
Strength"  using  the  formula 

"+C12+G12+C22+G22+C36+G36+ ( (C29+G29) *D41) . " 

This  formula  isolates  and  sums  the  "Ability  to  Contribute" 
area  strength  ratings,  applies  the  "Strength  Matching"  weight 
to  that  sum,  and  then  adds  in  the  remaining  area  strength 
ratings.   By  doing  so,  the  formula  is  responsible  for  the 
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downward  skewing  noticed  by  the  IP  Office  when  they  first 
applied  the  revised  evaluation  criteria. 

D.   APPLYING  THE  SPREADSHEET  CALCULATIONS  TO  THOSE  CANDIDATE 
PROGRAMS  NEVER  EVALUATED  WITH  REVISED  CRITERIA  AND  METHOD 

The  spreadsheet  program  representing  the  intent  of  the 
revised  criteria  and  procedure  was  then  applied  to  the  twelve 
programs  never  re- evaluated  with  the  revised  criteria. 
Criteria  scores  for  the  two  line  items  ("Workshare"  and 
"Payoff")  not  on  the  initial  evaluation  sheet  had  to  be 
developed.  For  "Workshare, "  a  value  equivalent  to  the  score 
given  to  "Industrial  Base  Impact"  on  the  initial  evaluation 
was  used.  For  "Payoff,"  the  subjectively  determined  "Payoff 
Potential"  from  the  initial  evaluation  was  translated  into  a 
numerical  score  between  one  and  ten  and  used  in  the 
spreadsheet.  The  "Degree  of  Convergence"  weight  assigned  to 
the  "Ability  to  Contribute"  area  on  the  initial  evaluation  was 
used  as  the  "Strength  Matching"  weight  of  the  spreadsheet. 
The  "Workshare"  and  "Strength  Matching"  assignments  were  done 
after  consulting  with  Mr.  Frank  Kenlon  of  the  IP  Office;  the 
"Payoff"  assignment  represents  a  natural  progression  from  one 
format  to  another.  The  results  of  this  analysis  are  shown  in 
the  table  below: 
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PROGRAM 
SHORT  TITLE 

STRENGTH 
WEIGHTING 

COMPOSITE 

STRENGTH 

RATING 

RSU 

1.25 

76.25 

AES 

1.25 

69.06 

TLSS 

1.25 

58.63 

DTS 

1.00 

53.90 

TTS 

1.25 

53.25 

OCA-G 

1.00 

50.35 

OCA-M 

1.00 

50.35 

OC-J 

1.00 

48.50 

OC-N 

1.00 

48.50 

TSS 

0.75 

41.03 

MWSS 

1.00 

39.20 

RSR 

1.00 

38.70 

E.   FINAL  PRIORITY  RANKING  OF  ALL  22  CANDIDATE  PROGRAMS 

The  following  table  represents  the  final  ranking  of  the  22 
programs  based  on  the  composite  strength  ratings  calculated  by 
the  spreadsheet;  included  is  a  column  with  the  current  status 
of  each  candidate  program: 


PROGRAM 
SHORT  TITLE 

COMPOSITE 

STRENGTH 

RATING 

CURRENT 
STATUS 

NMMS 

97.80 

SIGNED  MOU 

ASMCM 

86.18 

AWAITING  START 
OF  NEGOTIATIONS 

SPIN 

80.00 

IN  NEGOTIATION 

RSU 

76.25 

PROPOSAL 
WITHDRAWN 
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PROGRAM 
SHORT  TITLE 

COMPOSITE 

STRENGTH 

RATING 

CURRENT 
STATUS 

CSNI 

72.50 

SIGNED  MOU 

AES 

69.06 

PROPOSAL 
WITHDRAWN 

ADV  ST 

67.66 

DEVELOPING  RAN 

MUAV 

62.73 

DEVELOPING  RAN 

CIP 

62.10 

AWAITING  START 
OF  NEGOTIATIONS 

C2P2 

60.50 

DEVELOPING  RAN 

TLSS 

58.63 

PROPOSAL 
WITHDRAWN 

AL  BAT 

55.38 

DEVELOPING  RAN 

DTS 

53.90 

PROPOSAL 
WITHDRAWN 

TTS 

53.25 

PROPOSAL 
WITHDRAWN 

OCA-G 

50.35 

PROPOSAL 
WITHDRAWN 

OCA-M 

50.35 

PROPOSAL 
WITHDRAWN 

OC-J 

48.50 

PROPOSAL 
WITHDRAWN 

OC-N 

48.50 

PROPOSAL 
WITHDRAWN 

TSS 

41.03 

PROPOSAL 
WITHDRAWN 

RAIDS 

39.25 

PROPOSAL 
WITHDRAWN 

MWSS 

39.20 

PROPOSAL 
WITHDRAWN 

RSR 

38.70 

PROPOSAL 
WITHDRAWN 
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With  the  spreadsheet  evaluation,  it  appears  that  a 
composite  strength  score  of  55  or  above  represents  a 
threshold;  all  programs  that  did  not  reach  that  threshold 
resulted  in  the  candidate  program  being  withdrawn  from 
consideration.  Two  programs  (RSU  and  AES)  that  exceeded  the 
threshold  also  had  their  proposals  withdrawn;  to  correlate 
their  withdrawal  to  their  evaluation  scores,  statistical 
analysis  techniques  were  applied  to  the  spreadsheet  evaluation 
scores . 

F.   STATISTICAL  ANALYSIS  OF  SPREADSHEET  EVALUATION  SCORES 

Based  on  the  status  of  each  of  the  22  programs,  the 
programs  were  assigned  to  one  of  two  groups,  Unsuccessful  or 
Successful,  to  perform  a  statistical  analysis  of  the 
spreadsheet  evaluation  scores.  The  Unsuccessful  group 
consisted  of  those  programs  with  a  status  indicating  that  the 
proposal  had  been  withdrawn;  the  Successful  group  consisted  of 
all  others. 

Combined  (U.  S.  and  Foreign)  median  and  sample  mean  scores 
for  each  of  the  four  evaluation  areas  of  the  spreadsheet 
(Level  of  Commitment,  Programmatic  Factors,  Ability  to 
Contribute,  and  Technology/Security)  were  calculated  for  each 
group;  the  standard  distribution  of  the  mean  was  also 
calculated.  The  following  four  tables  record  the  results  of 
those  calculations: 
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COMBINED  LEVEL 
OF  COMMITMENT 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

3.50 

4.63 

SAMPLE  MEAN 

3.08 

5.54 

STANDARD 
DEVIATION 

0.88 

2.00 

COMBINED 

PROGRAMMATIC 

FACTORS 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

15.00 

20.70 

SAMPLE  MEAN 

15.44 

19.50 

STANDARD 
DEVIATION 

2.47 

3.87 

COMBINED 
ABILITY  TO 
CONTRIBUTE 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

13.00 

17.00 

SAMPLE  MEAN 

13.12 

16.67 

STANDARD 
DEVIATION 

2.36 

1.71 

COMBINED 

TECHNOLOGY/ 

SECURITY 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

20.00 

17.50 

SAMPLE  MEAN 

16.73 

17.00 

STANDARD 
DEVIATION 

4.38 

2.49 

Of  all  the  median  and  sample  means  calculated,  the  most 
statistically  significant,  in  terms  of  the  range  of  score 
disparity  between  the  Unsuccessful  and  Successful  groups,  are 
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the  sample  mean  scores  of  the  Level  of  Commitment  area. 
Combined  median,  sample  means,  and  standard  distributions  were 
calculated  for  each  line  item  in  the  Level  of  Commitment  area; 
the  results  are  shown  in  the  three  tables  below: 


COMBINED 

CONGRESS/ 

LEGISLATURE 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

2.00 

2.00 

SAMPLE  MEAN 

2.00 

2.22 

STANDARD 
DEVIATION 

0.00 

1.09 

COMBINED 
OSD/MOD 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

2.00 

8.00 

SAMPLE  MEAN 

2.00 

7.33 

STANDARD 
DEVIATION 

0.00 

4.09 

COMBINED 
SERVICE/ 
EQUIVALENT 

UNSUCCESSFUL 
PROPOSALS 

SUCCESSFUL 
PROPOSALS 

MEDIAN 

10.00 

12.00 

SAMPLE  MEAN 

8.46 

11.44 

STANDARD 
DEVIATION 

3.38 

4.69 

Statistically,  the  most  significant  of  these  line  item 
scores  are  the  median  and  sample  mean  values  of  the  combined 
OSD/MOD  line  item  scores.  Without  OSD/MOD  support,  a  proposal 
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appears  doomed;  with  OSD/MOD  support,  a  proposal's  weaknesses 
in  other  areas  may  be  surmounted. 

G.   CORRELATION  OF  IPO'S  EVALUATION  CRITERIA  WITH  OSD  SUCCESS 
INDICATORS 

At  a  Four  Powers  (United  States,  United  Kingdom,  France, 
Germany)  Conference  held  in  Monterey,  California  on  October  5, 
1991,  the  Deputy  Under  Secretary  of  Defense  for  International 
Programs  (DUSD(IP))  briefed  the  participants  on  OSD's 
viewpoint  regarding  the  criteria  of  success  for  cooperative 
programs;  the  OSD  criteria  are  presented  below: 

•  A  common,  well  defined  requirement; 

•  An  early  start  supported  by  technology  cooperation  and 
venture  capital  from  Nunn  amendment  funding; 

•  A  clear  view  ahead  on  life-cycle  funding; 

•  A  clear  view  of  competitive  programs; 

•  Equal  priority  for  each  player; 

•  Two  or  three  partners  maximum; 

•  Each  partner  bringing  something  to  the  program  besides 
money ; 

•  The  will  to  see  it  through,  including  a  mutual 
understanding  of  cultures,  concurrence  on  acquisition 
strategy,  strong  program  managers,  and  support  at  the 
highest  level. 

The  policy,  procedures  and  criteria  implemented  by  IPO 
through  the  evaluation  of  candidate  programs  generally 
supports  the  OSD  success  criteria.  One  instance  of  divergence 
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pertains  to  the  number  of  partners  being  considered  for  a 
program;  IPO  does  not  take  into  account  a  candidate  program 
having  more  than  three  partners  while  conducting  its 
evaluation.  Unless  it  has  significant  OSD/MOD  support,  a 
program  with  an  excessive  number  of  partners  may  experience 
difficulties  obtaining  the  authority  to  negotiate  an 
international  agreement. 
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V.    CONCLUSIONS  AND  RECOMMENDATIONS 

A.   CONCLUSIONS 

The  examination  of  the  evaluation  performed  by  the  Navy 
International  Programs  Office  on  22  candidate  program 
proposals  led  to  the  following  conclusions: 

•  Upon  revising  the  success  criteria,  a  miscommunication 
resulted  in  evaluations  being  conducted  in  such  a  manner 
that  overemphasized  the  "Ability  to  Contribute"  area  of 
the  criteria  and  that  produced  inaccurate  "Composite 
Strength"  scores  and  ranking. 

•  The  single,  evaluation  criteria  line  item  that  principally 
determines  whether  a  candidate  program  gains  approval  to 
develop  the  Request  Authority  to  Negotiate  document  is  the 
level  of  OSD/MOD  support  of  the  program;  by  extension,  the 
single  principal  evaluation  area  is  the  "Level  of 
Commi tment "  area . 

•  The  International  Programs  Office's  policy,  procedures, 
and  evaluation  criteria  are  in  general  agreement  with 
their  OSD  counterparts;  however,  the  International 
Programs  Office  does  not  penalize  a  candidate  program  for 
having  an  excessive  number  of  partners. 


B.   RECOMMENDATIONS 

The  recommendations  derived  from  the  above  conclusions  are 
as  follows: 

•  The  International  Programs  Office  should  correct  the 
software  programming  error  that  produces  the  evaluation 
anomalies;  this  should  be  accomplished  by  implementing  the 
spreadsheet  "Composite  Strength"  formula  in  Section  IV- C 
of  this  thesis. 

•  The  International  Programs  Office  should  direct  its 
resources  at  those  candidate  programs  with  a  composite 
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strength  score  of  55  or  above,  and  at  all  candidate 
programs  with  a  combined  OSD/MOD  line  item  score  of  three 
or  above . 

•  The  International  Programs  Office  should  subtract  points 
from  the  composite  strength  scores  of  those  candidate 
programs  with  an  excessive  number  of  partners  before 
ranking  to  reflect  the  greater  risk  of  the  program. 

•  The  International  Programs  Office  should  continue  to 
critique  the  criteria  and  procedures  used  to  evaluate 
candidate  programs  to  stay  current  with  the  rapidly- 
changing  environment  of  the  international  arena. 
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